2019-12-09T09:17:10.100 INFO:root:teuthology version: 1.0.0-8b630f9 2019-12-09T09:17:10.101 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.6mJMNA.tmp -- --verbose --lock --description rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi --block --owner scheduled_ofriedma@teuthology --archive /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584247 2019-12-09T09:17:10.142 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:17:10.203 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584247 branch: wip-ofriedma-dynamic-disable description: rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: ofriedma@redhat.com first_in_suite: false job_id: '4584247' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-ofriedma-dynamic-disable ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 osd fast shutdown: false osd objectstore: filestore osd shutdown pgref assert: true osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 1351c819e1a9518cde4c8b12d645782009028360 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: notcmalloc sha1: 1351c819e1a9518cde4c8b12d645782009028360 openssl_keys: rgw.client.0: ca: root client: client.0 embed-key: true rgw.client.1: ca: root client: client.1 embed-key: true root: client: client.0 cn: teuthology install: - client.0 - client.1 key-type: rsa:4096 rgw: client.0: ssl certificate: rgw.client.0 client.1: ssl certificate: rgw.client.1 compression type: random ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true workunit: branch: wip-ofriedma-dynamic-disable sha1: 1351c819e1a9518cde4c8b12d645782009028360 owner: scheduled_ofriedma@teuthology priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 1351c819e1a9518cde4c8b12d645782009028360 suite: rgw suite_branch: wip-ofriedma-dynamic-disable suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 1351c819e1a9518cde4c8b12d645782009028360 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.13080 2019-12-09T09:17:10.203 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa; will attempt to use it 2019-12-09T09:17:10.204 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks 2019-12-09T09:17:10.204 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-12-09T09:17:10.206 INFO:teuthology.task.internal:Checking packages... 2019-12-09T09:17:10.219 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '1351c819e1a9518cde4c8b12d645782009028360' 2019-12-09T09:17:10.219 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-09T09:17:10.219 INFO:teuthology.packaging:ref: None 2019-12-09T09:17:10.219 INFO:teuthology.packaging:tag: None 2019-12-09T09:17:10.220 INFO:teuthology.packaging:branch: wip-ofriedma-dynamic-disable 2019-12-09T09:17:10.220 INFO:teuthology.packaging:sha1: 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:17:10.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-ofriedma-dynamic-disable 2019-12-09T09:17:10.372 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7900.g1351c81 2019-12-09T09:17:10.372 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-12-09T09:17:10.384 INFO:teuthology.task.internal:no buildpackages task found 2019-12-09T09:17:10.384 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-12-09T09:17:10.396 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-12-09T09:17:10.396 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:17:10.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:17:20.458 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:17:30.474 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:17:40.490 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:17:50.504 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:18:00.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:18:10.545 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:18:20.568 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:18:30.583 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:18:40.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:18:50.621 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:19:00.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:19:10.669 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:19:20.685 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:19:30.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:19:40.718 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:19:50.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:20:00.748 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:20:10.765 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:20:20.780 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:20:30.799 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:20:40.808 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:20:50.824 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:21:00.840 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:21:10.853 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:21:20.868 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:21:30.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:21:40.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:21:50.928 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:22:00.944 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:22:10.969 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:22:20.980 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:22:30.996 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:22:41.013 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:22:51.030 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:23:01.047 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:23:11.064 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:23:21.081 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:23:31.097 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:23:41.113 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:23:51.131 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:24:01.151 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:24:11.164 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:24:21.182 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:24:31.201 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:24:41.217 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:24:51.232 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:25:01.257 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:25:11.273 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:25:21.293 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:25:31.309 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:25:41.326 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:25:51.343 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:26:01.361 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:26:11.385 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:26:21.405 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:26:31.421 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_ofriedma@teuthology', 'description': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584247', 'machine_type': 'smithi'} 2019-12-09T09:26:31.437 DEBUG:teuthology.lock.ops:locked ubuntu@smithi161.front.sepia.ceph.com, ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:26:31.508 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T09:26:31.703 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 7.6 2019-12-09T09:26:31.706 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 7.6 2019-12-09T09:26:32.033 INFO:teuthology.orchestra.console:Power off smithi059 2019-12-09T09:26:32.033 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T09:26:32.054 INFO:teuthology.orchestra.console:Power off smithi161 2019-12-09T09:26:32.054 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T09:26:32.065 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:32.077 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:32.198 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2019-12-09T09:26:32.369 INFO:teuthology.orchestra.console:Power on smithi059 2019-12-09T09:26:32.369 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T09:26:32.378 INFO:teuthology.orchestra.console:Power on smithi161 2019-12-09T09:26:32.378 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T09:26:32.391 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:32.402 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:36.415 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:36.434 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:36.663 INFO:teuthology.orchestra.console:Power on for smithi161 completed 2019-12-09T09:26:36.764 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2019-12-09T09:26:40.528 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:26:40.739 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2019-12-09T09:26:40.840 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2019-12-09T09:29:11.840 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:20.909 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:22.920 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:29.965 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:31.978 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:39.025 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:41.035 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:48.119 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:50.090 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:57.197 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:29:59.146 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:30:08.205 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:30:17.265 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:03.238 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:03.465 INFO:teuthology.orchestra.run.smithi059:> hostname 2019-12-09T09:31:04.026 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2019-12-09T09:31:04.027 INFO:teuthology.orchestra.run.smithi059:> grep smithi059.front.sepia.ceph.com /etc/hosts 2019-12-09T09:31:04.114 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2019-12-09T09:31:04.115 INFO:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2019-12-09T09:31:04.260 INFO:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hosts 2019-12-09T09:31:04.367 INFO:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/172.21.15.59/172.21.15.59/g' /etc/hosts 2019-12-09T09:31:04.472 INFO:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hostname 2019-12-09T09:31:04.560 INFO:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2019-12-09T09:31:04.651 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2019-12-09T09:31:04.653 DEBUG:teuthology.parallel:result is None 2019-12-09T09:31:23.277 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:23.529 INFO:teuthology.orchestra.run.smithi161:> hostname 2019-12-09T09:31:24.108 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com 2019-12-09T09:31:24.109 INFO:teuthology.orchestra.run.smithi161:> grep smithi161.front.sepia.ceph.com /etc/hosts 2019-12-09T09:31:24.194 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161 2019-12-09T09:31:24.195 INFO:teuthology.orchestra.run.smithi161:> sudo hostname smithi161 2019-12-09T09:31:24.340 INFO:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hosts 2019-12-09T09:31:24.426 INFO:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts 2019-12-09T09:31:24.501 INFO:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hostname 2019-12-09T09:31:24.566 INFO:teuthology.orchestra.run.smithi161:> sudo hostname smithi161 2019-12-09T09:31:24.632 INFO:teuthology.provision.fog.smithi161:Deploy complete! 2019-12-09T09:31:24.634 DEBUG:teuthology.parallel:result is None 2019-12-09T09:31:24.805 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2019-12-09T09:31:24.805 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T09:31:24.835 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com 2019-12-09T09:31:24.835 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T09:31:24.870 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2019-12-09T09:31:24.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:31:24.871 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:25.007 INFO:teuthology.orchestra.run.smithi059:> true 2019-12-09T09:31:25.408 INFO:teuthology.orchestra.run.smithi059:> uname -m 2019-12-09T09:31:25.477 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2019-12-09T09:31:25.477 INFO:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2019-12-09T09:31:25.540 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Linux" 2019-12-09T09:31:25.540 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="7 (Core)" 2019-12-09T09:31:25.540 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="7" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:31:25.541 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:31:25.542 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:25.543 INFO:teuthology.orchestra.run.smithi059.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:31:25.543 INFO:teuthology.orchestra.run.smithi059.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:31:25.543 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:31:25.544 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:31:25.544 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:25.545 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2019-12-09T09:31:25.558 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server 2019-12-09T09:31:25.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:31:25.559 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:25.648 INFO:teuthology.orchestra.run.smithi161:> true 2019-12-09T09:31:26.043 INFO:teuthology.orchestra.run.smithi161:> uname -m 2019-12-09T09:31:26.118 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2019-12-09T09:31:26.119 INFO:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2019-12-09T09:31:26.190 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Linux" 2019-12-09T09:31:26.191 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="7 (Core)" 2019-12-09T09:31:26.191 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2019-12-09T09:31:26.192 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:31:26.192 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="7" 2019-12-09T09:31:26.192 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:31:26.192 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2019-12-09T09:31:26.193 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:31:26.193 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:31:26.193 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:31:26.193 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:26.194 INFO:teuthology.orchestra.run.smithi161.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:31:26.194 INFO:teuthology.orchestra.run.smithi161.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:31:26.194 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:31:26.194 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:31:26.194 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:26.196 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2019-12-09T09:31:26.223 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-12-09T09:31:26.259 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6AGl/ohYvVLjyWTLFhWYFlZ+ZYWOB5t3S49q+vH9Mjic+B4zvNAQCGAZYRxb3T1qeLcX0nEns8q+Cu5vuSMMzUYuEU4y8Y/Rw/qIrqcFozyMiozUP68cGa2aqPR0MDwJRjCsi9o/ZFTv8mcX9ki1vio+dkOS1azdeZuriM27jEOVSS7torHGoOeymFDuHIvPa+LSZOGIRwvIfkFqCtsO3m4yjh0VcRzTu/QkIS4XEQRnsmDUpxC18LyuvxQ2WnLqt0iWFe4GzS+4UDXNuO25eo0FTlcPPwCmI927A+9KTCb3GwOTWW31nd2bIc85cyUqt2Vk+C7n1uGBIP4W4DeiL smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKKvsQpzhSuyM8U0q9CYbJsswotvPUE+t2JZkTvdffgHWi1jrLr6eF/BfCXio+oj+x3/o/rMvt/ABpUKakneYi8dioxoa+vCX0fPWpRs6mG0OvuZijylJLyOw2h8Oq2FgljuFJev584sCWJAnm5Br35IKwbM3RcagvELsxjp8XHQJz4YdRq/o5kylw0vvslYHRwooXOFr/+Nqzx1HTOc4nhSPO23R/aoTOiBlzCf4HEES/uMPg58PvzqLq6NY7Aq4Hj6SF9wl2/xleNDpgFSU9GgWKyrdMRKXVhEorvvkzLIXM8bVmfrmagij8+myNiVRZ8KoBjgXfw5N6i58bmw4X 2019-12-09T09:31:26.259 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:31:26.307 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-12-09T09:31:26.315 INFO:teuthology.task.internal:Saving configuration 2019-12-09T09:31:26.336 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-12-09T09:31:26.374 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-12-09T09:31:26.387 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 09:26:31.427524', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi059.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5bY5lyvMzmNwIkDTM57M/hYLUJ3bgczXjECFYKxEK19LY/ywpbcWXJ/tuVwWhR68TXjXwADQErukO30y40RKY=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584247'} 2019-12-09T09:31:26.400 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 09:26:31.427544', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi161.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQKrVRV7s2omlM35C9WyXppd3u57Z4HqKY0R+W92fSBtQ3+KDCo73427WKfr5EKlqorcfyTJsGq6vQxCjFQZL8=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584247'} 2019-12-09T09:31:26.401 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-12-09T09:31:26.417 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T09:31:26.417 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T09:31:26.417 INFO:teuthology.run_tasks:Running task console_log... 2019-12-09T09:31:26.489 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T09:31:26.490 INFO:teuthology.run_tasks:Running task internal.connect... 2019-12-09T09:31:26.500 INFO:teuthology.task.internal:Opening connections... 2019-12-09T09:31:26.500 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:31:26.501 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:26.597 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:31:26.598 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:31:26.694 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-12-09T09:31:26.704 INFO:teuthology.orchestra.run.smithi059:> uname -m 2019-12-09T09:31:26.945 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2019-12-09T09:31:26.946 INFO:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2019-12-09T09:31:27.019 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Linux" 2019-12-09T09:31:27.031 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="7 (Core)" 2019-12-09T09:31:27.032 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2019-12-09T09:31:27.032 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="7" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:31:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:31:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:31:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:31:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:31:27.034 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:27.035 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2019-12-09T09:31:27.048 INFO:teuthology.orchestra.run.smithi161:> uname -m 2019-12-09T09:31:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2019-12-09T09:31:27.167 INFO:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2019-12-09T09:31:27.250 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Linux" 2019-12-09T09:31:27.250 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="7 (Core)" 2019-12-09T09:31:27.250 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2019-12-09T09:31:27.250 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:31:27.250 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="7" 2019-12-09T09:31:27.250 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:31:27.251 INFO:teuthology.orchestra.run.smithi161.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:31:27.252 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:31:27.252 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:31:27.252 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:27.252 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2019-12-09T09:31:27.266 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-12-09T09:31:27.288 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-12-09T09:31:27.298 INFO:teuthology.task.internal:Checking for old test directory... 2019-12-09T09:31:27.298 INFO:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2019-12-09T09:31:27.302 INFO:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2019-12-09T09:31:27.388 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-12-09T09:31:27.398 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-12-09T09:31:27.398 INFO:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2019-12-09T09:31:27.403 INFO:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2019-12-09T09:31:27.487 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-12-09T09:31:27.525 INFO:teuthology.run_tasks:Running task kernel... 2019-12-09T09:31:27.543 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-12-09T09:31:27.543 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-12-09T09:31:27.543 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:31:27.544 INFO:teuthology.orchestra.run.smithi059:> sudo package-cleanup -y --oldkernels 2019-12-09T09:31:28.453 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:31:29.317 INFO:teuthology.orchestra.run.smithi059.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:29.318 INFO:teuthology.orchestra.run.smithi059.stdout:Trying other mirror. 2019-12-09T09:31:29.318 INFO:teuthology.orchestra.run.smithi059.stdout:To address this issue please refer to the below wiki article 2019-12-09T09:31:29.318 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:29.318 INFO:teuthology.orchestra.run.smithi059.stdout:https://wiki.centos.org/yum-errors 2019-12-09T09:31:29.318 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:29.319 INFO:teuthology.orchestra.run.smithi059.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T09:31:29.319 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:29.474 INFO:teuthology.orchestra.run.smithi059.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:29.474 INFO:teuthology.orchestra.run.smithi059.stdout:Trying other mirror. 2019-12-09T09:31:29.570 INFO:teuthology.orchestra.run.smithi059.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:29.570 INFO:teuthology.orchestra.run.smithi059.stdout:Trying other mirror. 2019-12-09T09:31:29.709 INFO:teuthology.orchestra.run.smithi059.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:29.709 INFO:teuthology.orchestra.run.smithi059.stdout:Trying other mirror. 2019-12-09T09:31:29.862 INFO:teuthology.orchestra.run.smithi059.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:29.862 INFO:teuthology.orchestra.run.smithi059.stdout:Trying other mirror. 2019-12-09T09:31:30.464 INFO:teuthology.orchestra.run.smithi059.stdout:No old kernels to remove 2019-12-09T09:31:30.478 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:31:30.479 INFO:teuthology.orchestra.run.smithi161:> sudo package-cleanup -y --oldkernels 2019-12-09T09:31:31.424 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:31:32.379 INFO:teuthology.orchestra.run.smithi161.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:32.379 INFO:teuthology.orchestra.run.smithi161.stdout:Trying other mirror. 2019-12-09T09:31:32.379 INFO:teuthology.orchestra.run.smithi161.stdout:To address this issue please refer to the below wiki article 2019-12-09T09:31:32.379 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:32.380 INFO:teuthology.orchestra.run.smithi161.stdout:https://wiki.centos.org/yum-errors 2019-12-09T09:31:32.380 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:32.380 INFO:teuthology.orchestra.run.smithi161.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T09:31:32.380 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:31:32.525 INFO:teuthology.orchestra.run.smithi161.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:32.525 INFO:teuthology.orchestra.run.smithi161.stdout:Trying other mirror. 2019-12-09T09:31:32.617 INFO:teuthology.orchestra.run.smithi161.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:32.617 INFO:teuthology.orchestra.run.smithi161.stdout:Trying other mirror. 2019-12-09T09:31:32.760 INFO:teuthology.orchestra.run.smithi161.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:32.760 INFO:teuthology.orchestra.run.smithi161.stdout:Trying other mirror. 2019-12-09T09:31:32.913 INFO:teuthology.orchestra.run.smithi161.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:31:32.913 INFO:teuthology.orchestra.run.smithi161.stdout:Trying other mirror. 2019-12-09T09:31:33.512 INFO:teuthology.orchestra.run.smithi161.stdout:No old kernels to remove 2019-12-09T09:31:33.533 INFO:teuthology.orchestra.run.smithi059:> uname -r 2019-12-09T09:31:33.627 INFO:teuthology.orchestra.run.smithi059.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:31:33.628 INFO:teuthology.task.kernel:Running kernel on smithi059: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:31:33.628 INFO:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2019-12-09T09:31:33.932 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:31:33.977 INFO:teuthology.orchestra.run.smithi059.stdout:Determining fastest mirrors 2019-12-09T09:31:34.110 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:31:34.110 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T09:31:34.111 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:31:34.112 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:31:35.734 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:31:35.734 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:31:35.735 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T09:31:37.210 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:31:37.991 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:37.992 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:31:37.999 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:38.000 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:31:38.000 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:31:38.000 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:31:38.000 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:31:38.000 INFO:teuthology.orchestra.run.smithi059.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 50 M 2019-12-09T09:31:38.001 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 64 M 2019-12-09T09:31:38.002 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:31:38.003 INFO:teuthology.orchestra.run.smithi059.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:31:39.099 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:31:39.293 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:31:40.090 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:31:40.090 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:32:17.123 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:32:17.627 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:32:17.628 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:32:17.628 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:32:17.628 INFO:teuthology.orchestra.run.smithi059.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T09:32:17.628 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:32:17.628 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:32:17.682 INFO:teuthology.orchestra.run.smithi059:> rpm -q kernel --last | head -n 1 2019-12-09T09:32:18.034 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:31:42 AM UTC 2019-12-09T09:32:18.036 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:32:18.036 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:32:18.037 INFO:teuthology.orchestra.run.smithi161:> uname -r 2019-12-09T09:32:18.154 INFO:teuthology.orchestra.run.smithi161.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:32:18.155 INFO:teuthology.task.kernel:Running kernel on smithi161: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:32:18.155 INFO:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2019-12-09T09:32:18.526 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:32:18.600 INFO:teuthology.orchestra.run.smithi161.stdout:Determining fastest mirrors 2019-12-09T09:32:18.734 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:32:18.734 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T09:32:18.734 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:32:18.735 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:32:20.307 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:32:20.307 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:32:20.307 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T09:32:21.582 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:32:22.389 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:32:22.390 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:32:22.390 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:32:22.391 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:32:22.392 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:32:22.392 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2019-12-09T09:32:22.392 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:32:22.393 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 50 M 2019-12-09T09:32:22.393 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 64 M 2019-12-09T09:32:22.393 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:32:22.393 INFO:teuthology.orchestra.run.smithi161.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:32:23.324 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:32:23.514 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:32:24.322 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:32:24.323 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:32:59.963 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:33:00.543 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:33:00.543 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:33:00.543 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:33:00.544 INFO:teuthology.orchestra.run.smithi161.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T09:33:00.544 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:33:00.544 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:33:00.583 INFO:teuthology.orchestra.run.smithi161:> rpm -q kernel --last | head -n 1 2019-12-09T09:33:00.972 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:32:27 AM UTC 2019-12-09T09:33:00.974 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:00.974 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:00.974 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-12-09T09:33:00.975 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T09:33:00.975 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T09:33:00.975 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-12-09T09:33:00.975 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T09:33:00.975 INFO:teuthology.orchestra.run.smithi059:> rpm -q kernel --last | head -n 1 2019-12-09T09:33:01.349 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:31:42 AM UTC 2019-12-09T09:33:01.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:01.352 INFO:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2 2019-12-09T09:33:01.525 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.02 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 0.76.el7.centos 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Group : System Environment/Base 2019-12-09T09:33:01.526 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 0 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:Relocations : (not relocatable) 2019-12-09T09:33:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : CentOS BuildSystem 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T09:33:01.528 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T09:33:01.529 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T09:33:01.529 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2019-12-09T09:33:01.530 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T09:33:01.530 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:01.530 INFO:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T09:33:02.036 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2019-12-09T09:33:02.190 INFO:teuthology.orchestra.run.smithi059.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T09:33:02.566 INFO:teuthology.orchestra.run.smithi059.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:33:02.598 INFO:teuthology.orchestra.run.smithi059.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T09:33:02.642 INFO:teuthology.orchestra.run.smithi059.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T09:33:02.667 INFO:teuthology.orchestra.run.smithi059.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T09:33:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T09:33:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T09:33:03.921 INFO:teuthology.orchestra.run.smithi059.stderr:done 2019-12-09T09:33:03.923 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:33:03.994 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.GSU8NXFSU6 2019-12-09T09:33:03.994 INFO:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GSU8NXFSU6 2019-12-09T09:33:04.077 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.GSU8NXFSU6 2019-12-09T09:33:04.303 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.GSU8NXFSU6 is 6KB 2019-12-09T09:33:04.411 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.GSU8NXFSU6 2019-12-09T09:33:04.500 INFO:teuthology.orchestra.run.smithi059:> sudo grub2-set-default 0 2019-12-09T09:33:04.699 INFO:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2019-12-09T09:33:04.704 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-12-09T09:33:04.705 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T09:33:04.705 INFO:teuthology.orchestra.run.smithi161:> rpm -q kernel --last | head -n 1 2019-12-09T09:33:05.067 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:32:27 AM UTC 2019-12-09T09:33:05.069 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:05.070 INFO:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2 2019-12-09T09:33:05.186 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2 2019-12-09T09:33:05.186 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.02 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 0.76.el7.centos 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:Group : System Environment/Base 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 0 2019-12-09T09:33:05.187 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Relocations : (not relocatable) 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : CentOS BuildSystem 2019-12-09T09:33:05.188 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : CentOS 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T09:33:05.189 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T09:33:05.190 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2019-12-09T09:33:05.191 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T09:33:05.191 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:05.191 INFO:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T09:33:05.825 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2019-12-09T09:33:05.953 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:33:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T09:33:06.234 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:33:06.263 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T09:33:06.303 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T09:33:06.320 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T09:33:06.351 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T09:33:06.361 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T09:33:07.499 INFO:teuthology.orchestra.run.smithi161.stderr:done 2019-12-09T09:33:07.502 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:33:07.635 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.3vlfxXJ3XP 2019-12-09T09:33:07.636 INFO:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3vlfxXJ3XP 2019-12-09T09:33:07.747 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.3vlfxXJ3XP 2019-12-09T09:33:07.930 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.3vlfxXJ3XP is 6KB 2019-12-09T09:33:07.999 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.3vlfxXJ3XP 2019-12-09T09:33:08.060 INFO:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 0 2019-12-09T09:33:08.309 INFO:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2019-12-09T09:33:08.313 INFO:teuthology.misc:Re-opening connections... 2019-12-09T09:33:08.313 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:08.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:08.314 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:08.316 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:33:08.316 INFO:teuthology.misc:trying to connect to ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:33:08.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:08.318 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:08.413 INFO:teuthology.orchestra.run.smithi161:> true 2019-12-09T09:33:08.741 DEBUG:teuthology.misc:waited 0.427922010422 2019-12-09T09:33:09.746 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:09.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:09.747 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:44.248 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:33:44.250 DEBUG:teuthology.misc:waited 35.9364910126 2019-12-09T09:33:45.251 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:45.251 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:45.252 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:48.316 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:33:48.317 DEBUG:teuthology.misc:waited 40.0034890175 2019-12-09T09:33:49.317 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:49.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:49.318 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:51.384 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:33:51.385 DEBUG:teuthology.misc:waited 43.0714991093 2019-12-09T09:33:52.386 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:52.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:52.387 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:54.456 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:33:54.457 DEBUG:teuthology.misc:waited 46.1434979439 2019-12-09T09:33:55.457 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:55.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:55.458 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:33:57.528 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:33:57.529 DEBUG:teuthology.misc:waited 49.2155220509 2019-12-09T09:33:58.529 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:33:58.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:33:58.530 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:00.600 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:00.601 DEBUG:teuthology.misc:waited 52.2874250412 2019-12-09T09:34:01.601 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:01.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:01.603 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:03.672 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:03.673 DEBUG:teuthology.misc:waited 55.3595490456 2019-12-09T09:34:04.673 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:04.674 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:04.674 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:06.744 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:06.745 DEBUG:teuthology.misc:waited 58.4315960407 2019-12-09T09:34:07.745 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:07.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:07.747 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:39.289 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:39.289 DEBUG:teuthology.misc:waited 90.9758090973 2019-12-09T09:34:40.290 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:40.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:40.291 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:40.293 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:40.293 DEBUG:teuthology.misc:waited 91.979681015 2019-12-09T09:34:41.294 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:41.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:41.294 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:41.295 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:41.295 DEBUG:teuthology.misc:waited 92.9822950363 2019-12-09T09:34:42.296 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:42.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:42.297 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:42.299 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:42.299 DEBUG:teuthology.misc:waited 93.98594594 2019-12-09T09:34:43.300 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:43.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:43.301 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:43.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2019-12-09T09:34:43.302 DEBUG:teuthology.misc:waited 94.988628149 2019-12-09T09:34:44.303 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:44.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:44.303 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:44.406 INFO:teuthology.orchestra.run.smithi059:> true 2019-12-09T09:34:44.749 DEBUG:teuthology.misc:waited 96.4355909824 2019-12-09T09:34:45.750 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-12-09T09:34:45.750 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:34:45.751 INFO:teuthology.orchestra.run.smithi059:> uname -r 2019-12-09T09:34:45.820 INFO:teuthology.orchestra.run.smithi059.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:34:45.821 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:34:45.821 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T09:34:45.821 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T09:34:45.821 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:34:45.822 INFO:teuthology.orchestra.run.smithi161:> uname -r 2019-12-09T09:34:52.989 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-12-09T09:34:52.989 ERROR:teuthology.task.kernel:Saw exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 695, in wait_for_reboot assert not need_to_install(ctx, client, need_install[client]), \ File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 184, in need_to_install stdout=uname_fp, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 428, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 98, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 493, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/transport.py", line 806, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/transport.py", line 933, in open_channel raise e error: [Errno 104] Connection reset by peer 2019-12-09T09:34:54.000 INFO:teuthology.misc:Re-opening connections... 2019-12-09T09:34:54.000 INFO:teuthology.misc:trying to connect to ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:34:54.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:54.002 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:54.091 INFO:teuthology.orchestra.run.smithi059:> true 2019-12-09T09:34:54.396 DEBUG:teuthology.misc:waited 0.395826816559 2019-12-09T09:34:55.397 INFO:teuthology.misc:trying to connect to ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:34:55.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:34:55.397 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:34:55.480 INFO:teuthology.orchestra.run.smithi161:> true 2019-12-09T09:34:55.796 DEBUG:teuthology.misc:waited 1.79622197151 2019-12-09T09:34:56.797 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T09:34:56.798 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:34:56.798 INFO:teuthology.orchestra.run.smithi161:> uname -r 2019-12-09T09:34:56.872 INFO:teuthology.orchestra.run.smithi161.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:34:56.873 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:34:56.873 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T09:34:57.876 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-12-09T09:34:57.876 INFO:teuthology.orchestra.run.smithi059:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T09:34:58.022 INFO:teuthology.orchestra.run.smithi059.stdout:ttyS1 2019-12-09T09:34:58.038 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-12-09T09:34:58.039 INFO:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T09:34:58.137 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2019-12-09T09:34:58.151 INFO:teuthology.run_tasks:Running task internal.base... 2019-12-09T09:34:58.164 INFO:teuthology.task.internal:Creating test directory... 2019-12-09T09:34:58.164 INFO:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T09:34:58.169 INFO:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T09:34:58.241 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-12-09T09:34:58.252 INFO:teuthology.run_tasks:Running task internal.archive... 2019-12-09T09:34:58.263 INFO:teuthology.task.internal:Creating archive directory... 2019-12-09T09:34:58.263 INFO:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T09:34:58.267 INFO:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T09:34:58.359 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-12-09T09:34:58.381 INFO:teuthology.task.internal:Enabling coredump saving... 2019-12-09T09:34:58.381 INFO:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:34:58.385 INFO:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:34:58.438 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:34:58.499 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:34:58.500 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-12-09T09:34:58.518 INFO:teuthology.task.internal:Configuring sudo... 2019-12-09T09:34:58.518 INFO:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T09:34:58.521 INFO:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T09:34:58.633 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-12-09T09:34:58.643 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-12-09T09:34:58.643 INFO:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T09:34:58.647 INFO:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T09:34:58.728 INFO:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:34:58.821 INFO:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:34:59.069 INFO:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:34:59.181 INFO:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:34:59.296 INFO:teuthology.orchestra.run.smithi059:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T09:34:59.397 INFO:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:34:59.454 INFO:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:34:59.541 INFO:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:34:59.599 INFO:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:34:59.676 INFO:teuthology.orchestra.run.smithi161:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T09:34:59.732 INFO:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2019-12-09T09:34:59.735 INFO:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2019-12-09T09:34:59.852 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T09:34:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T09:34:59.925 INFO:teuthology.run_tasks:Running task internal.timer... 2019-12-09T09:34:59.982 INFO:teuthology.task.internal:Starting timer... 2019-12-09T09:34:59.982 INFO:teuthology.run_tasks:Running task pcp... 2019-12-09T09:35:00.035 INFO:teuthology.run_tasks:Running task selinux... 2019-12-09T09:35:00.047 INFO:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2019-12-09T09:35:00.212 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: [ OK ] 2019-12-09T09:35:00.215 INFO:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2019-12-09T09:35:00.317 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: [ OK ] 2019-12-09T09:35:00.319 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-12-09T09:35:00.319 INFO:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2019-12-09T09:35:00.416 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2019-12-09T09:35:00.416 INFO:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2019-12-09T09:35:00.468 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2019-12-09T09:35:00.468 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi059.front.sepia.ceph.com': 'permissive'} 2019-12-09T09:35:00.468 INFO:teuthology.orchestra.run.smithi059:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T09:35:00.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:35:00.589 INFO:teuthology.orchestra.run.smithi161:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T09:35:00.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:35:00.648 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-12-09T09:35:00.649 INFO:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2019-12-09T09:35:00.779 INFO:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2019-12-09T09:35:00.841 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-12-09T09:35:00.857 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-12-09T09:35:00.857 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-12-09T09:35:00.879 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-12-09T09:35:00.884 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-12-09T09:35:01.332 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-12-09T09:35:02.586 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2019-12-09T09:35:05.225 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-12-09T09:35:05.251 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:05 +0000 (0:00:00.064) 0:00:00.064 ******* 2019-12-09T09:35:05.261 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:05 +0000 (0:00:00.012) 0:00:00.077 ******* 2019-12-09T09:35:08.072 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2019-12-09T09:35:08.073 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] 2019-12-09T09:35:08.182 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:08 +0000 (0:00:02.921) 0:00:02.998 ******* 2019-12-09T09:35:08.639 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] 2019-12-09T09:35:08.698 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-12-09T09:35:08.700 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:35:08.737 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:08 +0000 (0:00:00.553) 0:00:03.552 ******* 2019-12-09T09:35:08.771 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:08 +0000 (0:00:00.034) 0:00:03.587 ******* 2019-12-09T09:35:09.053 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:35:09.090 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:09 +0000 (0:00:00.318) 0:00:03.906 ******* 2019-12-09T09:35:09.405 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-12-09T09:35:09.406 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T09:35:09.448 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:09 +0000 (0:00:00.358) 0:00:04.264 ******* 2019-12-09T09:35:09.488 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [ansible-managed : Create the ansible user.] ****************************** ok: [smithi059.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T09:35:09.537 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:09 +0000 (0:00:00.081) 0:00:04.346 ******* 2019-12-09T09:35:09.585 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:09 +0000 (0:00:00.054) 0:00:04.401 ******* 2019-12-09T09:35:09.591 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-12-09T09:35:09.593 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:09.645 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:09 +0000 (0:00:00.059) 0:00:04.460 ******* 2019-12-09T09:35:10.105 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:35:10.157 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:10 +0000 (0:00:00.510) 0:00:04.971 ******* 2019-12-09T09:35:10.198 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:10 +0000 (0:00:00.042) 0:00:05.014 ******* 2019-12-09T09:35:11.705 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-12-09T09:35:11.706 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884110.17-8700134310942/source", "state": "file", "uid": 0} 2019-12-09T09:35:11.752 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:11 +0000 (0:00:01.553) 0:00:06.567 ******* 2019-12-09T09:35:11.766 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884110.22-32167728638186/source", "state": "file", "uid": 0} 2019-12-09T09:35:11.822 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:11 +0000 (0:00:00.070) 0:00:06.638 ******* 2019-12-09T09:35:12.298 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2019-12-09T09:35:12.298 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:12.364 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:12.619 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:12.718 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:12.956 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:13.063 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:13.252 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:13.399 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:13.587 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:13.749 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:13.920 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:14.058 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:14.254 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:14.416 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:35:14.550 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2019-12-09T09:35:14.556 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-12-09T09:35:14.820 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:14 +0000 (0:00:02.997) 0:00:09.636 ******* 2019-12-09T09:35:14.854 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:14 +0000 (0:00:00.033) 0:00:09.670 ******* 2019-12-09T09:35:15.749 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi059.front.sepia.ceph.com] 2019-12-09T09:35:15.800 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] 2019-12-09T09:35:15.847 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-12-09T09:35:15.849 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:15 +0000 (0:00:00.993) 0:00:10.663 ******* 2019-12-09T09:35:15.924 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:15 +0000 (0:00:00.076) 0:00:10.740 ******* 2019-12-09T09:35:16.410 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-12-09T09:35:16.411 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.222519", "end": "2019-12-09 09:35:16.385591", "rc": 0, "start": "2019-12-09 09:35:16.163072", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T09:35:16.450 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:16 +0000 (0:00:00.525) 0:00:11.265 ******* 2019-12-09T09:35:16.457 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.254971", "end": "2019-12-09 09:35:16.408172", "rc": 0, "start": "2019-12-09 09:35:16.153201", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T09:35:16.507 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:16 +0000 (0:00:00.055) 0:00:11.323 ******* 2019-12-09T09:35:16.766 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-09T09:35:16.767 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008518", "end": "2019-12-09 09:35:16.742965", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:35:16.734447", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T09:35:16.806 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:16 +0000 (0:00:00.298) 0:00:11.621 ******* 2019-12-09T09:35:16.832 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-09T09:35:16.834 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:16.879 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:16 +0000 (0:00:00.073) 0:00:11.695 ******* 2019-12-09T09:35:16.885 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-09T09:35:16.885 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.015427", "end": "2019-12-09 09:35:16.259766", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:35:16.244339", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T09:35:16.932 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:16 +0000 (0:00:00.052) 0:00:11.747 ******* 2019-12-09T09:35:16.937 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-09T09:35:16.938 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:16.980 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:16 +0000 (0:00:00.048) 0:00:11.796 ******* 2019-12-09T09:35:16.986 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-09T09:35:16.987 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.029 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.048) 0:00:11.845 ******* 2019-12-09T09:35:17.037 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-09T09:35:17.038 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.080 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.048) 0:00:11.893 ******* 2019-12-09T09:35:17.083 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-09T09:35:17.084 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.126 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.048) 0:00:11.941 ******* 2019-12-09T09:35:17.131 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-09T09:35:17.132 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.175 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.049) 0:00:11.991 ******* 2019-12-09T09:35:17.181 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-09T09:35:17.183 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.223 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.047) 0:00:12.039 ******* 2019-12-09T09:35:17.229 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-09T09:35:17.230 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.283 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.060) 0:00:12.099 ******* 2019-12-09T09:35:17.295 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-09T09:35:17.297 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.334 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.050) 0:00:12.149 ******* 2019-12-09T09:35:17.360 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-09T09:35:17.361 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:17.398 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.064) 0:00:12.214 ******* 2019-12-09T09:35:17.629 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-12-09T09:35:17.631 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:35:17.674 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.275) 0:00:12.489 ******* 2019-12-09T09:35:17.718 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:35:17.765 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:17 +0000 (0:00:00.091) 0:00:12.581 ******* 2019-12-09T09:35:18.090 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T09:35:18.091 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T09:35:18.116 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T09:35:18.393 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T09:35:18.446 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T09:35:18.755 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T09:35:18.798 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T09:35:19.105 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T09:35:19.117 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T09:35:19.447 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T09:35:19.462 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T09:35:19.783 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T09:35:19.806 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T09:35:20.156 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T09:35:20.183 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T09:35:20.454 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T09:35:20.535 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T09:35:20.772 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T09:35:20.893 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T09:35:21.101 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T09:35:21.250 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T09:35:21.453 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T09:35:21.615 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T09:35:21.762 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T09:35:21.985 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T09:35:22.111 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T09:35:22.320 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T09:35:22.446 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T09:35:22.669 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T09:35:22.818 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T09:35:22.988 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T09:35:23.161 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T09:35:23.334 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T09:35:23.509 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T09:35:23.662 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T09:35:23.815 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T09:35:24.020 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T09:35:24.139 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T09:35:24.394 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T09:35:24.499 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T09:35:24.756 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T09:35:24.843 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T09:35:25.097 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T09:35:25.099 INFO:teuthology.task.ansible.out: 2019-12-09T09:35:25.171 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T09:35:25.459 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T09:35:25.503 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T09:35:25.782 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T09:35:25.899 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T09:35:26.156 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T09:35:26.213 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T09:35:26.508 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T09:35:26.550 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T09:35:26.873 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T09:35:26.875 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T09:35:27.217 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T09:35:27.232 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T09:35:27.573 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T09:35:27.575 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T09:35:27.893 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T09:35:27.895 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T09:35:28.198 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T09:35:28.298 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T09:35:28.531 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T09:35:28.588 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T09:35:28.827 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T09:35:28.904 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T09:35:29.147 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T09:35:29.279 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T09:35:29.492 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T09:35:29.621 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T09:35:29.816 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T09:35:29.987 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T09:35:30.102 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T09:35:30.334 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T09:35:30.431 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T09:35:30.694 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T09:35:30.821 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T09:35:31.053 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T09:35:31.191 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:35:31.371 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T09:35:31.565 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T09:35:31.705 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:35:31.884 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T09:35:32.049 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T09:35:32.202 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T09:35:32.386 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T09:35:32.579 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T09:35:32.716 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T09:35:32.853 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T09:35:33.049 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T09:35:33.193 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T09:35:33.383 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T09:35:33.490 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T09:35:33.721 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T09:35:33.827 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:35:34.077 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T09:35:34.118 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T09:35:34.392 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:35:34.425 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T09:35:34.755 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T09:35:34.756 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T09:35:35.078 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T09:35:35.112 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T09:35:35.387 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T09:35:35.457 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T09:35:35.725 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T09:35:35.820 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T09:35:36.030 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T09:35:36.148 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T09:35:36.303 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T09:35:36.502 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T09:35:36.633 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T09:35:36.853 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T09:35:36.970 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T09:35:37.228 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T09:35:37.354 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T09:35:37.580 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T09:35:37.686 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T09:35:37.929 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T09:35:38.028 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T09:35:38.283 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T09:35:38.358 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T09:35:38.640 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T09:35:38.724 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T09:35:38.995 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T09:35:39.081 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T09:35:39.338 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T09:35:39.433 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:35:39.647 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T09:35:39.798 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T09:35:40.008 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T09:35:40.153 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T09:35:40.343 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:35:40.496 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T09:35:40.704 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T09:35:40.862 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T09:35:41.041 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T09:35:41.220 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T09:35:41.377 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T09:35:41.547 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T09:35:41.719 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T09:35:41.865 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T09:35:42.087 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T09:35:42.193 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T09:35:42.421 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T09:35:42.538 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T09:35:42.754 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T09:35:42.881 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T09:35:43.124 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T09:35:43.232 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T09:35:43.471 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T09:35:43.563 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T09:35:43.826 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T09:35:43.901 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T09:35:44.183 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T09:35:44.274 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T09:35:44.523 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T09:35:44.599 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T09:35:44.870 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T09:35:44.922 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T09:35:45.238 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T09:35:45.243 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T09:35:45.548 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T09:35:45.619 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T09:35:45.891 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T09:35:45.966 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T09:35:46.251 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T09:35:46.322 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T09:35:46.632 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T09:35:46.666 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T09:35:47.007 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T09:35:47.044 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T09:35:47.351 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T09:35:47.363 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T09:35:47.686 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T09:35:47.687 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T09:35:48.016 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T09:35:48.061 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T09:35:48.343 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T09:35:48.396 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T09:35:48.698 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T09:35:48.730 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T09:35:49.039 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T09:35:49.085 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T09:35:49.392 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T09:35:49.431 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T09:35:49.720 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T09:35:49.795 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T09:35:50.058 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T09:35:50.144 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T09:35:50.402 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T09:35:50.527 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T09:35:50.728 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T09:35:50.896 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T09:35:51.028 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T09:35:51.269 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T09:35:51.333 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T09:35:51.617 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T09:35:51.664 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T09:35:51.943 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T09:35:52.019 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T09:35:52.284 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T09:35:52.359 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:35:52.615 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T09:35:52.685 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T09:35:52.993 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T09:35:53.355 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T09:35:53.409 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T09:35:53.607 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:53 +0000 (0:00:35.836) 0:00:48.418 ******* 2019-12-09T09:35:53.628 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T09:35:53.665 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:53 +0000 (0:00:00.063) 0:00:48.481 ******* 2019-12-09T09:35:53.713 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T09:35:53.717 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T09:35:53.718 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T09:35:53.720 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T09:35:53.721 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T09:35:53.724 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-09T09:35:53.725 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-09T09:35:53.726 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 2019-12-09T09:35:53.732 INFO:teuthology.task.ansible.out:6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f8 2019-12-09T09:35:53.733 INFO:teuthology.task.ansible.out:48846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8a 2019-12-09T09:35:53.735 INFO:teuthology.task.ansible.out:a01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalho 2019-12-09T09:35:53.736 INFO:teuthology.task.ansible.out:jr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T09:35:53.776 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:53 +0000 (0:00:00.110) 0:00:48.592 ******* 2019-12-09T09:35:53.783 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T09:35:53.784 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:35:54.102 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T09:35:54.784 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-09T09:35:54.785 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T09:35:54.829 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:54 +0000 (0:00:01.052) 0:00:49.644 ******* 2019-12-09T09:35:54.837 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ changed: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T09:35:54.966 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:54 +0000 (0:00:00.137) 0:00:49.782 ******* 2019-12-09T09:35:54.994 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T09:35:55.033 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:55 +0000 (0:00:00.065) 0:00:49.848 ******* 2019-12-09T09:35:55.084 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T09:35:55.089 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T09:35:55.097 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T09:35:55.099 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T09:35:55.100 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T09:35:55.103 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJy 2019-12-09T09:35:55.112 INFO:teuthology.task.ansible.out:w 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 2019-12-09T09:35:55.114 INFO:teuthology.task.ansible.out: 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf853 2019-12-09T09:35:55.126 INFO:teuthology.task.ansible.out:2237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36b 2019-12-09T09:35:55.129 INFO:teuthology.task.ansible.out:deb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a28 2019-12-09T09:35:55.134 INFO:teuthology.task.ansible.out:73481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c 2019-12-09T09:35:55.137 INFO:teuthology.task.ansible.out:4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T09:35:55.178 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:55 +0000 (0:00:00.145) 0:00:49.994 ******* 2019-12-09T09:35:55.241 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T09:35:55.601 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T09:35:55.915 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi059.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T09:35:55.967 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:35:55 +0000 (0:00:00.789) 0:00:50.783 ******* 2019-12-09T09:35:55.974 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:35:55.974 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T09:35:56.338 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T09:35:56.391 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validat 2019-12-09T09:35:56.393 INFO:teuthology.task.ansible.out:e_certs": true} 2019-12-09T09:35:56.706 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c1931 2019-12-09T09:35:56.708 INFO:teuthology.task.ansible.out:69dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2019-12-09T09:35:56.772 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T09:35:57.066 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T09:35:57.140 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T09:35:57.396 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T09:35:57.397 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T09:35:57.510 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T09:35:57.745 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T09:35:57.861 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T09:35:57.862 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T09:35:58.124 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T09:35:58.167 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:58.232 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T09:35:58.508 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T09:35:58.509 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T09:35:58.519 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:58.530 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:58.541 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:58.605 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T09:35:58.607 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T09:35:58.859 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T09:35:58.946 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T09:35:59.225 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c63 2019-12-09T09:35:59.228 INFO:teuthology.task.ansible.out:74d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgn 2019-12-09T09:35:59.230 INFO:teuthology.task.ansible.out:VY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T09:35:59.298 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T09:35:59.320 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:59.572 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T09:35:59.680 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T09:35:59.681 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T09:35:59.702 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:59.724 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:59.724 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:35:59.884 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T09:36:00.077 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T09:36:00.225 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T09:36:00.437 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T09:36:00.439 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T09:36:00.542 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T09:36:00.789 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T09:36:00.894 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T09:36:01.132 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T09:36:01.260 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T09:36:01.518 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T09:36:01.592 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T09:36:01.594 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T09:36:01.870 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T09:36:01.927 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T09:36:01.929 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T09:36:02.189 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T09:36:02.280 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T09:36:02.543 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T09:36:02.630 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T09:36:02.922 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T09:36:02.923 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T09:36:03.008 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T09:36:03.010 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T09:36:03.277 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T09:36:03.278 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T09:36:03.378 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T09:36:03.613 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T09:36:03.710 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T09:36:03.952 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T09:36:04.052 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T09:36:04.310 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T09:36:04.312 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T09:36:04.385 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T09:36:04.666 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T09:36:04.743 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T09:36:05.018 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T09:36:05.093 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T09:36:05.095 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T09:36:05.348 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T09:36:05.428 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T09:36:05.722 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T09:36:05.807 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T09:36:06.086 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T09:36:06.162 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T09:36:06.454 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T09:36:06.456 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T09:36:06.560 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T09:36:06.814 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T09:36:06.900 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T09:36:07.152 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T09:36:07.248 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T09:36:07.526 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T09:36:07.603 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:36:07.866 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T09:36:07.942 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T09:36:08.186 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T09:36:08.272 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T09:36:08.541 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T09:36:08.616 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T09:36:08.618 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T09:36:08.889 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:36:08.986 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T09:36:08.988 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T09:36:09.246 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T09:36:09.312 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T09:36:09.604 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T09:36:09.668 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T09:36:09.942 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T09:36:09.944 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T09:36:10.017 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T09:36:10.018 INFO:teuthology.task.ansible.out: 2019-12-09T09:36:10.286 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T09:36:10.287 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T09:36:10.373 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T09:36:10.375 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:36:10.628 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T09:36:10.706 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T09:36:10.707 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T09:36:10.709 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T09:36:10.982 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T09:36:11.057 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T09:36:11.311 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T09:36:11.406 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T09:36:11.671 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T09:36:11.672 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:36:11.730 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T09:36:12.033 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T09:36:12.034 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T09:36:12.036 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T09:36:12.055 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T09:36:12.385 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T09:36:12.428 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T09:36:12.756 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T09:36:12.830 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T09:36:13.116 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T09:36:13.194 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T09:36:13.470 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T09:36:13.536 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T09:36:13.835 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T09:36:13.879 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T09:36:13.881 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T09:36:14.175 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T09:36:14.217 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T09:36:14.517 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T09:36:14.570 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T09:36:14.865 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T09:36:14.908 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T09:36:14.910 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T09:36:15.214 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQW 2019-12-09T09:36:15.216 INFO:teuthology.task.ansible.out:VyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gabrioux", "validate_certs": true} 2019-12-09T09:36:15.257 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T09:36:15.563 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T09:36:15.598 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T09:36:15.944 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T09:36:15.965 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T09:36:16.284 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:36:16.309 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T09:36:16.311 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T09:36:16.615 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T09:36:16.708 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T09:36:16.957 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T09:36:17.081 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T09:36:17.292 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T09:36:17.428 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T09:36:17.640 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T09:36:17.774 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:36:17.972 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T09:36:18.129 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T09:36:18.298 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T09:36:18.504 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T09:36:18.642 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T09:36:18.854 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T09:36:18.998 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T09:36:19.199 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T09:36:19.337 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T09:36:19.565 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T09:36:19.638 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T09:36:19.941 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T09:36:19.994 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T09:36:20.301 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T09:36:20.354 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T09:36:20.652 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T09:36:20.695 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T09:36:20.975 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T09:36:21.031 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T09:36:21.352 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T09:36:21.399 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T09:36:21.696 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T09:36:21.728 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T09:36:22.048 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T09:36:22.070 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T09:36:22.384 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T09:36:22.396 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T09:36:22.721 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T09:36:22.743 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T09:36:23.092 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T09:36:23.153 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T09:36:23.446 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T09:36:23.489 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T09:36:23.816 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T09:36:23.832 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T09:36:23.833 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T09:36:24.160 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T09:36:24.180 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T09:36:24.181 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T09:36:24.517 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T09:36:24.539 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T09:36:24.869 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T09:36:24.891 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T09:36:25.232 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T09:36:25.248 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T09:36:25.584 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T09:36:25.586 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T09:36:25.606 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T09:36:25.941 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T09:36:25.942 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T09:36:25.951 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T09:36:26.299 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T09:36:26.303 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T09:36:26.649 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T09:36:26.652 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T09:36:26.965 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T09:36:27.018 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T09:36:27.296 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T09:36:27.371 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T09:36:27.640 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T09:36:27.743 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T09:36:27.984 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T09:36:28.101 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T09:36:28.334 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-09T09:36:28.456 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T09:36:28.688 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T09:36:28.825 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T09:36:29.029 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T09:36:29.188 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T09:36:29.383 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-09T09:36:29.384 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:36:29.531 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T09:36:29.720 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T09:36:29.900 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T09:36:30.062 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T09:36:30.165 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:30 +0000 (0:00:34.197) 0:01:24.981 ******* 2019-12-09T09:36:30.213 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:30.214 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.224 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.236 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.247 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.248 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:30.249 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9 2019-12-09T09:36:30.250 INFO:teuthology.task.ansible.out:c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.260 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.271 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.282 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.294 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.597 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:30.599 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T09:36:30.655 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:30.657 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T09:36:30.667 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:30.954 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:30.955 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T09:36:31.050 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:31.052 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T09:36:31.317 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:31.319 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-09T09:36:31.320 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:36:31.398 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:31.400 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T09:36:31.662 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:31.664 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T09:36:31.767 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:31.769 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T09:36:31.779 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.790 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.802 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.813 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.814 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.836 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.847 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.848 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.859 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.870 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.871 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.882 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.896 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.898 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.909 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.920 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.931 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.942 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.953 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.964 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.965 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.976 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:31.998 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.000 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.001 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:32.003 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T09:36:32.004 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.048 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:36:32.090 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:32.092 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.093 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.095 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.139 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:32 +0000 (0:00:01.973) 0:01:26.954 ******* 2019-12-09T09:36:32.145 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.146 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.148 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.149 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.151 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.165 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.166 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.167 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.177 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.178 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.189 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.190 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.201 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.202 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.203 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.219 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.225 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.236 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.237 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.248 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.248 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.250 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.263 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.283 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.294 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.307 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.316 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.327 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.328 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.339 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.350 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.361 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.362 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.377 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.381 INFO:teuthology.task.ansible.out: 2019-12-09T09:36:32.382 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.403 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.405 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.415 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.417 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.427 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.438 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.453 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.455 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.465 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.477 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.490 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.501 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.513 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.525 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.526 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.549 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.552 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.562 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.563 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.585 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.587 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.597 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.609 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.610 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T09:36:32.611 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.630 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.646 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.647 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.716 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:32 +0000 (0:00:00.576) 0:01:27.531 ******* 2019-12-09T09:36:32.743 INFO:teuthology.task.ansible.out: 2019-12-09T09:36:32.745 INFO:teuthology.task.ansible.out:TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:32.786 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:32 +0000 (0:00:00.070) 0:01:27.601 ******* 2019-12-09T09:36:32.966 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T09:36:33.157 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:33.158 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T09:36:33.300 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T09:36:33.499 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:33.500 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T09:36:33.635 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:33.636 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T09:36:33.646 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.657 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.668 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.682 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.683 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.694 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.705 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.716 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.727 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.738 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.749 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.761 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.782 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.783 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.794 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.805 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.819 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.830 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.831 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:33.832 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T09:36:33.845 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:36:33.847 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.857 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.868 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.879 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.880 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.891 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.904 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.923 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.924 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.935 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.946 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.957 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.968 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.979 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:33.990 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.001 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.001 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.013 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.023 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.037 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.038 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.049 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.060 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.076 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.077 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.087 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.098 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.110 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.121 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.132 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.133 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.147 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.152 INFO:teuthology.task.ansible.out: 2019-12-09T09:36:34.164 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.176 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.177 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi161.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T09:36:34.188 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.190 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.328 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.329 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.341 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.352 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.363 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.364 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.375 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.386 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.400 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.409 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.420 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.436 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.443 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.455 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.478 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.500 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.511 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.523 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi161.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.540 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.546 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.557 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.568 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.579 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.590 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.591 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.602 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.613 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.624 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.635 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.635 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.647 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.658 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.738 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:34 +0000 (0:00:01.952) 0:01:29.554 ******* 2019-12-09T09:36:34.765 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T09:36:34.767 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:34.813 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:34 +0000 (0:00:00.070) 0:01:29.625 ******* 2019-12-09T09:36:34.858 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:34.859 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T09:36:35.194 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T09:36:35.195 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T09:36:35.514 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T09:36:35.537 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T09:36:35.862 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T09:36:35.884 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T09:36:36.215 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T09:36:36.227 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T09:36:36.543 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T09:36:36.555 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T09:36:36.878 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T09:36:36.890 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T09:36:37.199 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T09:36:37.242 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T09:36:37.524 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T09:36:37.577 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T09:36:37.859 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T09:36:37.921 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T09:36:38.163 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T09:36:38.276 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T09:36:38.512 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T09:36:38.596 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T09:36:38.846 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T09:36:38.953 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T09:36:39.176 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T09:36:39.298 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T09:36:39.537 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T09:36:39.617 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T09:36:39.869 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T09:36:39.973 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T09:36:40.306 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T09:36:40.494 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T09:36:40.648 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T09:36:40.998 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T09:36:41.073 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T09:36:41.341 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T09:36:41.618 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T09:36:41.657 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T09:36:41.972 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T09:36:42.147 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T09:36:42.533 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T09:36:42.715 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T09:36:43.144 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T09:36:43.315 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T09:36:43.747 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T09:36:43.828 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T09:36:44.287 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T09:36:44.428 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T09:36:44.893 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T09:36:45.015 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T09:36:45.484 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T09:36:45.632 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T09:36:46.050 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T09:36:46.205 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T09:36:46.573 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T09:36:46.785 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T09:36:47.106 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T09:36:47.149 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T09:36:47.443 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T09:36:47.685 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T09:36:47.728 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T09:36:48.047 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T09:36:48.213 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T09:36:48.393 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T09:36:48.704 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T09:36:48.750 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T09:36:49.034 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T09:36:49.098 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T09:36:49.436 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T09:36:49.620 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T09:36:49.694 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:49 +0000 (0:00:14.885) 0:01:44.510 ******* 2019-12-09T09:36:49.786 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T09:36:50.107 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T09:36:50.464 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T09:36:50.797 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T09:36:50.798 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884209.72-46869207511850/source", "state": "file", "uid": 0} 2019-12-09T09:36:50.799 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:50.801 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T09:36:50.860 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:50 +0000 (0:00:01.162) 0:01:45.672 ******* 2019-12-09T09:36:50.885 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi161.front.sepia.ceph.com] => { "msg": "Host smithi161.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T09:36:50.945 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:50 +0000 (0:00:00.087) 0:01:45.760 ******* 2019-12-09T09:36:50.988 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T09:36:50.989 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T09:36:51.055 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.110) 0:01:45.871 ******* 2019-12-09T09:36:51.084 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T09:36:51.085 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:51.127 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.068) 0:01:45.939 ******* 2019-12-09T09:36:51.171 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:51.172 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T09:36:51.565 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-09T09:36:51.567 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.110901", "end": "2019-12-09 09:36:50.960395", "rc": 0, "start": "2019-12-09 09:36:50.849494", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T09:36:51.607 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.483) 0:01:46.422 ******* 2019-12-09T09:36:51.645 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T09:36:51.647 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:51.720 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.112) 0:01:46.535 ******* 2019-12-09T09:36:51.747 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:36:51.748 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T09:36:51.784 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T09:36:51.785 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:51.832 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.112) 0:01:46.648 ******* 2019-12-09T09:36:51.873 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.041) 0:01:46.689 ******* 2019-12-09T09:36:51.906 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T09:36:51.908 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:51.949 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:51 +0000 (0:00:00.071) 0:01:46.761 ******* 2019-12-09T09:36:51.981 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.045 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.099) 0:01:46.861 ******* 2019-12-09T09:36:52.082 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.118 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.072) 0:01:46.933 ******* 2019-12-09T09:36:52.144 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T09:36:52.146 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.186 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.068) 0:01:47.001 ******* 2019-12-09T09:36:52.226 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi161.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi161.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:52.274 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.088) 0:01:47.089 ******* 2019-12-09T09:36:52.301 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.344 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.070) 0:01:47.160 ******* 2019-12-09T09:36:52.373 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.420 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.075) 0:01:47.235 ******* 2019-12-09T09:36:52.447 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T09:36:52.448 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:52.490 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.070) 0:01:47.306 ******* 2019-12-09T09:36:52.518 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-09T09:36:52.519 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.073) 0:01:47.380 ******* 2019-12-09T09:36:52.601 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T09:36:52.603 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:52.655 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.083) 0:01:47.463 ******* 2019-12-09T09:36:52.685 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T09:36:52.687 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.746 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.098) 0:01:47.562 ******* 2019-12-09T09:36:52.786 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.829 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.082) 0:01:47.645 ******* 2019-12-09T09:36:52.857 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T09:36:52.859 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.902 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.069) 0:01:47.714 ******* 2019-12-09T09:36:52.936 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T09:36:52.937 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:52.993 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:52 +0000 (0:00:00.094) 0:01:47.809 ******* 2019-12-09T09:36:53.022 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T09:36:53.022 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:53.024 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T09:36:53.025 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884211.86-29587695682367/source", "state": "file", "uid": 0} 2019-12-09T09:36:53.062 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.069) 0:01:47.878 ******* 2019-12-09T09:36:53.100 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.037) 0:01:47.916 ******* 2019-12-09T09:36:53.106 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-09T09:36:53.108 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => { "msg": "Host smithi059.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T09:36:53.147 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.046) 0:01:47.963 ******* 2019-12-09T09:36:53.154 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T09:36:53.155 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.200 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.052) 0:01:48.015 ******* 2019-12-09T09:36:53.205 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T09:36:53.206 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T09:36:53.247 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.047) 0:01:48.063 ******* 2019-12-09T09:36:53.254 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-09T09:36:53.255 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.314 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.066) 0:01:48.130 ******* 2019-12-09T09:36:53.323 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T09:36:53.325 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.364 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.049) 0:01:48.180 ******* 2019-12-09T09:36:53.384 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.415 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.050) 0:01:48.231 ******* 2019-12-09T09:36:53.457 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.500 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.076) 0:01:48.308 ******* 2019-12-09T09:36:53.529 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T09:36:53.530 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.570 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.077) 0:01:48.385 ******* 2019-12-09T09:36:53.607 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T09:36:53.608 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.646 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.076) 0:01:48.462 ******* 2019-12-09T09:36:53.706 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-09T09:36:53.708 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:53.744 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.098) 0:01:48.560 ******* 2019-12-09T09:36:53.772 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T09:36:53.824 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.079) 0:01:48.639 ******* 2019-12-09T09:36:53.842 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-09T09:36:53.842 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.114702", "end": "2019-12-09 09:36:53.796574", "rc": 0, "start": "2019-12-09 09:36:53.681872", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T09:36:53.882 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.058) 0:01:48.698 ******* 2019-12-09T09:36:53.888 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T09:36:53.890 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.931 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.049) 0:01:48.747 ******* 2019-12-09T09:36:53.937 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T09:36:53.938 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:53.982 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:53 +0000 (0:00:00.046) 0:01:48.794 ******* 2019-12-09T09:36:54.019 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T09:36:54.021 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.063 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.085) 0:01:48.879 ******* 2019-12-09T09:36:54.102 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T09:36:54.103 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.155 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.091) 0:01:48.970 ******* 2019-12-09T09:36:54.183 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.224 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.064) 0:01:49.034 ******* 2019-12-09T09:36:54.245 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-09T09:36:54.246 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.284 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.064) 0:01:49.099 ******* 2019-12-09T09:36:54.289 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T09:36:54.290 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:36:54.334 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.050) 0:01:49.150 ******* 2019-12-09T09:36:54.341 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T09:36:54.342 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.399 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.064) 0:01:49.214 ******* 2019-12-09T09:36:54.440 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi059.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi059.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:54.486 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.087) 0:01:49.302 ******* 2019-12-09T09:36:54.516 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-09T09:36:54.520 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.556 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.069) 0:01:49.372 ******* 2019-12-09T09:36:54.595 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-09T09:36:54.596 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.635 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.078) 0:01:49.451 ******* 2019-12-09T09:36:54.673 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T09:36:54.675 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:54.713 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.078) 0:01:49.529 ******* 2019-12-09T09:36:54.765 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.808 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.091) 0:01:49.621 ******* 2019-12-09T09:36:54.840 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi059.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:54.884 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.078) 0:01:49.699 ******* 2019-12-09T09:36:54.912 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T09:36:54.914 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:54.949 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:54 +0000 (0:00:00.065) 0:01:49.765 ******* 2019-12-09T09:36:54.991 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T09:36:54.993 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.032 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.079) 0:01:49.844 ******* 2019-12-09T09:36:55.076 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T09:36:55.078 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.121 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.086) 0:01:49.931 ******* 2019-12-09T09:36:55.143 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T09:36:55.145 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.183 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.068) 0:01:49.999 ******* 2019-12-09T09:36:55.209 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T09:36:55.211 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:55.254 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.070) 0:01:50.069 ******* 2019-12-09T09:36:55.291 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T09:36:55.293 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.332 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.078) 0:01:50.147 ******* 2019-12-09T09:36:55.362 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-09T09:36:55.363 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.425 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.093) 0:01:50.241 ******* 2019-12-09T09:36:55.463 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-09T09:36:55.465 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.503 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.077) 0:01:50.319 ******* 2019-12-09T09:36:55.531 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.591 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.087) 0:01:50.406 ******* 2019-12-09T09:36:55.620 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T09:36:55.622 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T09:36:55.632 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T09:36:55.634 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.675 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.083) 0:01:50.490 ******* 2019-12-09T09:36:55.703 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T09:36:55.705 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.748 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.071) 0:01:50.562 ******* 2019-12-09T09:36:55.786 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi059.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:36:55.825 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.078) 0:01:50.640 ******* 2019-12-09T09:36:55.852 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T09:36:55.889 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.063) 0:01:50.704 ******* 2019-12-09T09:36:55.922 INFO:teuthology.task.ansible.out: 2019-12-09T09:36:55.924 INFO:teuthology.task.ansible.out:TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:36:55.963 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:55 +0000 (0:00:00.074) 0:01:50.778 ******* 2019-12-09T09:36:56.352 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T09:36:56.354 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:36:56.409 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:56 +0000 (0:00:00.446) 0:01:51.225 ******* 2019-12-09T09:36:56.563 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi161.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T09:36:56.604 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:56 +0000 (0:00:00.194) 0:01:51.420 ******* 2019-12-09T09:36:57.398 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T09:36:57.747 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:36:57.749 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884216.64-190667159494913/source", "state": "file", "uid": 0} 2019-12-09T09:36:58.335 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T09:36:58.336 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T09:36:58.390 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:58 +0000 (0:00:01.786) 0:01:53.206 ******* 2019-12-09T09:36:58.865 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:36:58.866 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884217.75-174184911654053/source", "state": "file", "uid": 0} 2019-12-09T09:36:58.926 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:58 +0000 (0:00:00.535) 0:01:53.742 ******* 2019-12-09T09:36:59.558 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884218.43-226627133554138/source", "state": "file", "uid": 0} 2019-12-09T09:36:59.589 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-09T09:36:59.591 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T09:36:59.908 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:36:59.909 INFO:teuthology.task.ansible.out: [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-12-09T09:36:59.948 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:36:59 +0000 (0:00:01.021) 0:01:54.764 ******* 2019-12-09T09:37:00.717 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:37:00.718 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575884219.6-46203938911877/source", "state": "file", "uid": 0} 2019-12-09T09:37:00.775 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:00 +0000 (0:00:00.827) 0:01:55.591 ******* 2019-12-09T09:37:01.249 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-09T09:37:01.251 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T09:37:01.576 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:37:01.626 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:01 +0000 (0:00:00.850) 0:01:56.442 ******* 2019-12-09T09:37:12.212 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:11.923104", "end": "2019-12-09 09:37:11.602699", "rc": 0, "start": "2019-12-09 09:36:59.679595", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.ash.fastserv.com\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.ash.fastserv.com", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T09:37:12.257 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:12 +0000 (0:00:10.630) 0:02:07.073 ******* 2019-12-09T09:37:12.738 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T09:37:12.739 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T09:37:13.027 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:37:13.068 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.810) 0:02:07.884 ******* 2019-12-09T09:37:13.225 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:37:13.227 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.272 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.203) 0:02:08.087 ******* 2019-12-09T09:37:13.548 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T09:37:13.549 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.588 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.316) 0:02:08.404 ******* 2019-12-09T09:37:13.614 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T09:37:13.616 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.671 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.083) 0:02:08.487 ******* 2019-12-09T09:37:13.698 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:37:13.699 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.743 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.071) 0:02:08.559 ******* 2019-12-09T09:37:13.769 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T09:37:13.771 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.809 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.065) 0:02:08.625 ******* 2019-12-09T09:37:13.837 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.874 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.065) 0:02:08.690 ******* 2019-12-09T09:37:13.900 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-09T09:37:13.902 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:13.943 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:13 +0000 (0:00:00.065) 0:02:08.756 ******* 2019-12-09T09:37:13.978 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T09:37:13.979 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:14.017 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:14 +0000 (0:00:00.077) 0:02:08.833 ******* 2019-12-09T09:37:14.140 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-12-09T09:37:14.141 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.179769", "end": "2019-12-09 09:37:14.106889", "rc": 0, "start": "2019-12-09 09:37:01.927120", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.fileplanet.com\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.fileplanet.com", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T09:37:14.184 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:14 +0000 (0:00:00.166) 0:02:09.000 ******* 2019-12-09T09:37:14.538 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T09:37:14.540 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T09:37:14.861 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:37:14.904 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:14 +0000 (0:00:00.719) 0:02:09.719 ******* 2019-12-09T09:37:14.933 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.012 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.108) 0:02:09.827 ******* 2019-12-09T09:37:15.038 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T09:37:15.040 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.079 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.067) 0:02:09.895 ******* 2019-12-09T09:37:15.106 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T09:37:15.108 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.146 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.066) 0:02:09.961 ******* 2019-12-09T09:37:15.177 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:37:15.179 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.217 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.070) 0:02:10.032 ******* 2019-12-09T09:37:15.244 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T09:37:15.246 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.282 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.065) 0:02:10.098 ******* 2019-12-09T09:37:15.308 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T09:37:15.310 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.347 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.064) 0:02:10.162 ******* 2019-12-09T09:37:15.373 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-09T09:37:15.374 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.418 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.071) 0:02:10.234 ******* 2019-12-09T09:37:15.444 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T09:37:15.446 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:15.486 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:15 +0000 (0:00:00.067) 0:02:10.302 ******* 2019-12-09T09:37:16.105 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-09T09:37:16.106 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.762752", "end": "2019-12-09 09:37:15.500444", "rc": 0, "start": "2019-12-09 09:37:13.737692", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:37:16.176 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:16 +0000 (0:00:00.690) 0:02:10.992 ******* 2019-12-09T09:37:16.536 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.708919", "end": "2019-12-09 09:37:16.500493", "rc": 0, "start": "2019-12-09 09:37:15.791574", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:37:16.581 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:16 +0000 (0:00:00.405) 0:02:11.397 ******* 2019-12-09T09:37:16.918 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T09:37:16.961 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:16 +0000 (0:00:00.379) 0:02:11.776 ******* 2019-12-09T09:37:16.994 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T09:37:16.995 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.032 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.071) 0:02:11.848 ******* 2019-12-09T09:37:17.058 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T09:37:17.060 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.097 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.065) 0:02:11.913 ******* 2019-12-09T09:37:17.114 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-09T09:37:17.114 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T09:37:17.153 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.055) 0:02:11.968 ******* 2019-12-09T09:37:17.158 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T09:37:17.160 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.203 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.050) 0:02:12.019 ******* 2019-12-09T09:37:17.209 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T09:37:17.209 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.253 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.049) 0:02:12.068 ******* 2019-12-09T09:37:17.258 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T09:37:17.259 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.299 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.045) 0:02:12.114 ******* 2019-12-09T09:37:17.306 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T09:37:17.308 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.352 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.048) 0:02:12.162 ******* 2019-12-09T09:37:17.374 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T09:37:17.375 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.414 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.066) 0:02:12.229 ******* 2019-12-09T09:37:17.441 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T09:37:17.442 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:17.481 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:17 +0000 (0:00:00.066) 0:02:12.296 ******* 2019-12-09T09:37:18.154 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-09T09:37:18.155 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T09:37:18.195 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:18 +0000 (0:00:00.714) 0:02:13.010 ******* 2019-12-09T09:37:18.388 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T09:37:18.428 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:18 +0000 (0:00:00.233) 0:02:13.244 ******* 2019-12-09T09:37:21.998 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-09T09:37:21.999 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T09:37:22.037 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:22 +0000 (0:00:03.608) 0:02:16.852 ******* 2019-12-09T09:37:22.345 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T09:37:22.386 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:22 +0000 (0:00:00.348) 0:02:17.201 ******* 2019-12-09T09:37:22.960 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi161.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T09:37:23.365 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T09:37:23.811 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T09:37:24.241 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T09:37:24.597 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T09:37:24.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:24 +0000 (0:00:02.256) 0:02:19.457 ******* 2019-12-09T09:37:24.931 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-09T09:37:24.932 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:37:24.977 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:24 +0000 (0:00:00.335) 0:02:19.793 ******* 2019-12-09T09:37:25.108 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T09:37:25.109 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T09:37:25.153 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:25 +0000 (0:00:00.175) 0:02:19.969 ******* 2019-12-09T09:37:25.476 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-09T09:37:25.477 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:37:25.514 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:25 +0000 (0:00:00.361) 0:02:20.330 ******* 2019-12-09T09:37:25.770 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:37:25.771 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T09:37:26.427 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T09:37:26.595 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T09:37:27.369 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T09:37:27.480 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T09:37:27.542 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:27 +0000 (0:00:02.027) 0:02:22.358 ******* 2019-12-09T09:37:27.570 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T09:37:27.571 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:27.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:27 +0000 (0:00:00.071) 0:02:22.430 ******* 2019-12-09T09:37:27.652 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T09:37:27.653 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:37:27.694 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:27 +0000 (0:00:00.079) 0:02:22.510 ******* 2019-12-09T09:37:28.039 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi161.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T09:37:28.079 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:28 +0000 (0:00:00.385) 0:02:22.895 ******* 2019-12-09T09:37:28.130 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T09:37:28.132 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.141 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.153 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.164 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.206 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:28 +0000 (0:00:00.126) 0:02:23.021 ******* 2019-12-09T09:37:28.280 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:37:28.282 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T09:37:28.322 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:28 +0000 (0:00:00.116) 0:02:23.138 ******* 2019-12-09T09:37:28.349 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T09:37:28.351 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.396 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:28 +0000 (0:00:00.074) 0:02:23.212 ******* 2019-12-09T09:37:28.434 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T09:37:28.436 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:37:28.470 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:28 +0000 (0:00:00.074) 0:02:23.286 ******* 2019-12-09T09:37:28.822 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-09T09:37:28.824 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T09:37:28.863 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:28 +0000 (0:00:00.392) 0:02:23.679 ******* 2019-12-09T09:37:28.922 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T09:37:28.923 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.945 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.957 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:28.969 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:37:29.006 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:29 +0000 (0:00:00.142) 0:02:23.821 ******* 2019-12-09T09:37:31.647 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T09:37:31.649 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T09:37:31.688 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:31 +0000 (0:00:02.682) 0:02:26.504 ******* 2019-12-09T09:37:31.726 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T09:37:31.728 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:31.773 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:31 +0000 (0:00:00.084) 0:02:26.588 ******* 2019-12-09T09:37:32.416 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T09:37:32.417 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T09:37:32.467 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:32 +0000 (0:00:00.694) 0:02:27.283 ******* 2019-12-09T09:37:32.508 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T09:37:32.509 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:32.552 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:32 +0000 (0:00:00.080) 0:02:27.364 ******* 2019-12-09T09:37:32.698 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T09:37:32.740 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:32 +0000 (0:00:00.191) 0:02:27.556 ******* 2019-12-09T09:37:33.695 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-09T09:37:33.696 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-09T09:37:33.698 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T09:37:33.734 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:33 +0000 (0:00:00.994) 0:02:28.550 ******* 2019-12-09T09:37:33.769 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:33 +0000 (0:00:00.035) 0:02:28.585 ******* 2019-12-09T09:37:33.796 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T09:37:33.797 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:33.835 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:33 +0000 (0:00:00.065) 0:02:28.651 ******* 2019-12-09T09:37:34.154 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T09:37:34.156 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:37:34.198 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:34 +0000 (0:00:00.363) 0:02:29.014 ******* 2019-12-09T09:37:34.558 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T09:37:34.559 INFO:teuthology.task.ansible.out:fatal: [smithi161.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T09:37:34.595 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:34 +0000 (0:00:00.396) 0:02:29.411 ******* 2019-12-09T09:37:34.632 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-09T09:37:34.633 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:34.670 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:34 +0000 (0:00:00.074) 0:02:29.486 ******* 2019-12-09T09:37:34.728 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-09T09:37:34.729 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T09:37:34.766 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:34 +0000 (0:00:00.096) 0:02:29.582 ******* 2019-12-09T09:37:34.798 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T09:37:34.800 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:34.838 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:34 +0000 (0:00:00.071) 0:02:29.653 ******* 2019-12-09T09:37:35.188 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T09:37:35.189 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:37:35.234 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:35 +0000 (0:00:00.396) 0:02:30.050 ******* 2019-12-09T09:37:35.610 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-09T09:37:35.612 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T09:37:35.657 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:35 +0000 (0:00:00.422) 0:02:30.473 ******* 2019-12-09T09:37:35.749 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T09:37:35.751 INFO:teuthology.task.ansible.out:fatal: [smithi059.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T09:37:35.836 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:35 +0000 (0:00:00.175) 0:02:30.649 ******* 2019-12-09T09:37:35.880 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:37:35.915 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:35 +0000 (0:00:00.082) 0:02:30.731 ******* 2019-12-09T09:37:36.544 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-09T09:37:36.546 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 09:34:53 UTC", "ActiveEnterTimestampMonotonic": "19818540", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice nss-lookup.target var-run.mount local-fs.target time-sync.target network.target systemd-journald.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 09:34:53 UTC", "AssertTimestampMonotonic": "19817995", "Before": "multi-user.target shutdown.target getty@tty1.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 09:34:53 UTC", "ConditionTimestampMonotonic": "19817993", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8969", "ExecMainStartTimestamp": "Mon 2019-12-09 09:34:53 UTC", "ExecMainStartTimestampMonotonic": "19818498", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-09T09:37:36.548 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-12-09 09:34:53 UTC] ; stop_time=[n/a] ; pid=8969 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 09:34:53 UTC", "InactiveExitTimestampMonotonic": "19818540", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8969", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-09T09:37:36.549 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 09:34:53 UTC", "WatchdogTimestampMonotonic": "19818523", "WatchdogUSec": "0"}} 2019-12-09T09:37:36.584 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:36 +0000 (0:00:00.669) 0:02:31.400 ******* 2019-12-09T09:37:36.890 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T09:37:36.924 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:36 +0000 (0:00:00.339) 0:02:31.739 ******* 2019-12-09T09:37:36.941 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T09:37:36.942 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.003882", "end": "2019-12-09 09:37:36.324838", "rc": 0, "start": "2019-12-09 09:37:36.320956", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T09:37:36.979 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:36 +0000 (0:00:00.055) 0:02:31.795 ******* 2019-12-09T09:37:37.373 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-09T09:37:37.375 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 09:34:43 UTC", "ActiveEnterTimestampMonotonic": "18799299", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target var-run.mount network.target local-fs.target systemd-journald.socket nss-lookup.target system.slice time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 09:34:43 UTC", "AssertTimestampMonotonic": "18798329", "Before": "xdm.service multi-user.target shutdown.target getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 09:34:43 UTC", "ConditionTimestampMonotonic": "18798328", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8730", "ExecMainStartTimestamp": "Mon 2019-12-09 09:34:43 UTC", "ExecMainStartTimestampMonotonic": "18799234", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-09T09:37:37.377 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-12-09 09:34:43 UTC] ; stop_time=[n/a] ; pid=8730 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 09:34:43 UTC", "InactiveExitTimestampMonotonic": "18799299", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8730", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-09T09:37:37.378 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 09:34:43 UTC", "WatchdogTimestampMonotonic": "18799274", "WatchdogUSec": "0"}} 2019-12-09T09:37:37.419 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:37 +0000 (0:00:00.440) 0:02:32.235 ******* 2019-12-09T09:37:37.459 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T09:37:37.460 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T09:37:37.511 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:37 +0000 (0:00:00.092) 0:02:32.327 ******* 2019-12-09T09:37:37.753 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T09:37:37.754 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004596", "end": "2019-12-09 09:37:37.722323", "rc": 0, "start": "2019-12-09 09:37:37.717727", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T09:37:37.813 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:37 +0000 (0:00:00.300) 0:02:32.628 ******* 2019-12-09T09:37:38.332 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T09:37:38.334 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T09:37:38.380 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:37:38 +0000 (0:00:00.562) 0:02:33.191 ******* 2019-12-09T09:38:04.109 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-09T09:38:04.110 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-09T09:38:04.111 INFO:teuthology.task.ansible.out: 20 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T09:38:04.149 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:04 +0000 (0:00:25.773) 0:02:58.964 ******* 2019-12-09T09:38:04.887 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-09T09:38:04.888 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T09:38:04.937 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:04 +0000 (0:00:00.785) 0:02:59.749 ******* 2019-12-09T09:38:06.219 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-09T09:38:06.220 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.fileplanet.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-09T09:38:06.221 INFO:teuthology.task.ansible.out: 16 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T09:38:06.280 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:06 +0000 (0:00:01.346) 0:03:01.096 ******* 2019-12-09T09:38:06.869 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-09T09:38:06.871 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T09:38:06.908 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:06 +0000 (0:00:00.627) 0:03:01.723 ******* 2019-12-09T09:38:13.035 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-09T09:38:13.035 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.769643", "end": "2019-12-09 09:38:12.429056", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:38:04.659413", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T09:38:13.091 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:13 +0000 (0:00:06.183) 0:03:07.907 ******* 2019-12-09T09:38:13.093 INFO:teuthology.task.ansible.out: 2019-12-09T09:38:14.005 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-09T09:38:14.006 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T09:38:14.045 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:14 +0000 (0:00:00.952) 0:03:08.859 ******* 2019-12-09T09:38:14.386 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-09T09:38:14.388 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.013842", "end": "2019-12-09 09:38:13.786921", "rc": 0, "start": "2019-12-09 09:38:13.773079", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T09:38:14.425 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:14 +0000 (0:00:00.381) 0:03:09.241 ******* 2019-12-09T09:38:14.775 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T09:38:14.777 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.017682", "end": "2019-12-09 09:38:14.171115", "rc": 0, "start": "2019-12-09 09:38:14.153433", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:38:14.815 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:14 +0000 (0:00:00.389) 0:03:09.631 ******* 2019-12-09T09:38:15.625 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-09T09:38:15.627 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.391479", "end": "2019-12-09 09:38:15.595126", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:38:07.203647", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T09:38:15.664 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:15 +0000 (0:00:00.848) 0:03:10.480 ******* 2019-12-09T09:38:16.644 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-09T09:38:16.646 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T09:38:16.684 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:16 +0000 (0:00:01.019) 0:03:11.499 ******* 2019-12-09T09:38:17.030 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-09T09:38:17.031 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.018596", "end": "2019-12-09 09:38:17.005278", "rc": 0, "start": "2019-12-09 09:38:16.986682", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T09:38:17.072 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:17 +0000 (0:00:00.387) 0:03:11.887 ******* 2019-12-09T09:38:17.415 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T09:38:17.417 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.018506", "end": "2019-12-09 09:38:17.387717", "rc": 0, "start": "2019-12-09 09:38:17.369211", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:38:17.452 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:17 +0000 (0:00:00.380) 0:03:12.268 ******* 2019-12-09T09:38:23.131 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.967041", "end": "2019-12-09 09:38:22.519951", "rc": 0, "start": "2019-12-09 09:38:14.552910", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:38:23.169 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:23 +0000 (0:00:05.717) 0:03:17.985 ******* 2019-12-09T09:38:23.497 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T09:38:23.539 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:23 +0000 (0:00:00.369) 0:03:18.354 ******* 2019-12-09T09:38:23.577 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T09:38:23.578 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:38:23.620 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:23 +0000 (0:00:00.081) 0:03:18.436 ******* 2019-12-09T09:38:23.658 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T09:38:23.659 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:38:23.723 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:23 +0000 (0:00:00.102) 0:03:18.538 ******* 2019-12-09T09:38:23.770 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T09:38:23.771 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T09:38:23.810 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:23 +0000 (0:00:00.087) 0:03:18.626 ******* 2019-12-09T09:38:23.848 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T09:38:23.850 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:38:23.885 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:23 +0000 (0:00:00.074) 0:03:18.701 ******* 2019-12-09T09:38:24.207 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-09T09:38:24.209 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:38:24.247 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:24 +0000 (0:00:00.361) 0:03:19.063 ******* 2019-12-09T09:38:24.598 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T09:38:24.600 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T09:38:24.636 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:24 +0000 (0:00:00.389) 0:03:19.452 ******* 2019-12-09T09:38:24.664 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T09:38:24.666 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:24.722 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:24 +0000 (0:00:00.084) 0:03:19.537 ******* 2019-12-09T09:38:24.748 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T09:38:24.749 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:24.786 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:24 +0000 (0:00:00.064) 0:03:19.602 ******* 2019-12-09T09:38:25.165 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-09T09:38:25.167 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T09:38:25.202 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:25 +0000 (0:00:00.416) 0:03:20.018 ******* 2019-12-09T09:38:26.220 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T09:38:26.222 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T09:38:26.260 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:26 +0000 (0:00:01.058) 0:03:21.076 ******* 2019-12-09T09:38:26.297 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T09:38:26.299 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi161"}, "changed": false} TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.498134", "end": "2019-12-09 09:38:26.262646", "rc": 0, "start": "2019-12-09 09:38:17.764512", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:38:26.336 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:26 +0000 (0:00:00.076) 0:03:21.152 ******* 2019-12-09T09:38:26.379 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:26 +0000 (0:00:00.042) 0:03:21.195 ******* 2019-12-09T09:38:26.719 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-09T09:38:26.720 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T09:38:26.819 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:26 +0000 (0:00:00.439) 0:03:21.635 ******* 2019-12-09T09:38:26.850 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T09:38:26.851 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:38:26.900 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:26 +0000 (0:00:00.080) 0:03:21.716 ******* 2019-12-09T09:38:26.940 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi059.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:38:27.007 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:27 +0000 (0:00:00.106) 0:03:21.822 ******* 2019-12-09T09:38:27.059 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T09:38:27.060 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T09:38:27.099 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:27 +0000 (0:00:00.092) 0:03:21.915 ******* 2019-12-09T09:38:27.137 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T09:38:27.139 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:38:27.191 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:27 +0000 (0:00:00.088) 0:03:22.004 ******* 2019-12-09T09:38:27.366 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-09T09:38:27.368 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi161.front.sepia.ceph.com", "ansible_hostname": "smithi161", "ansible_nodename": "smithi161"}, "changed": true, "name": "smithi161"} 2019-12-09T09:38:27.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:27 +0000 (0:00:00.216) 0:03:22.220 ******* 2019-12-09T09:38:27.526 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-09T09:38:27.528 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:38:27.568 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:27 +0000 (0:00:00.163) 0:03:22.384 ******* 2019-12-09T09:38:27.903 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T09:38:27.905 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T09:38:27.953 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:27 +0000 (0:00:00.377) 0:03:22.761 ******* 2019-12-09T09:38:27.973 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T09:38:27.974 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:28.040 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:28 +0000 (0:00:00.088) 0:03:22.850 ******* 2019-12-09T09:38:28.064 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T09:38:28.066 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:28.103 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:28 +0000 (0:00:00.068) 0:03:22.919 ******* 2019-12-09T09:38:28.331 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-09T09:38:28.331 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T09:38:28.370 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:28 +0000 (0:00:00.266) 0:03:23.186 ******* 2019-12-09T09:38:28.484 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-09T09:38:28.485 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T09:38:28.524 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:28 +0000 (0:00:00.153) 0:03:23.339 ******* 2019-12-09T09:38:29.329 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T09:38:29.330 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T09:38:29.374 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:29 +0000 (0:00:00.850) 0:03:24.190 ******* 2019-12-09T09:38:29.607 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T09:38:29.647 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:29 +0000 (0:00:00.272) 0:03:24.463 ******* 2019-12-09T09:38:29.686 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T09:38:29.688 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi059"}, "changed": false} 2019-12-09T09:38:29.728 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:29 +0000 (0:00:00.080) 0:03:24.544 ******* 2019-12-09T09:38:30.253 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi161.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T09:38:30.291 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:30 +0000 (0:00:00.562) 0:03:25.106 ******* 2019-12-09T09:38:30.300 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-09T09:38:30.300 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi059.front.sepia.ceph.com", "ansible_hostname": "smithi059", "ansible_nodename": "smithi059"}, "changed": true, "name": "smithi059"} 2019-12-09T09:38:30.338 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:30 +0000 (0:00:00.047) 0:03:25.154 ******* 2019-12-09T09:38:31.237 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T09:38:31.239 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T09:38:31.277 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:31 +0000 (0:00:00.938) 0:03:26.092 ******* 2019-12-09T09:38:31.293 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-09T09:38:31.294 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T09:38:31.332 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:31 +0000 (0:00:00.055) 0:03:26.148 ******* 2019-12-09T09:38:31.645 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:38:31.646 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:38:31.694 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:31 +0000 (0:00:00.362) 0:03:26.510 ******* 2019-12-09T09:38:32.033 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:38:32.034 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T09:38:32.083 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:32 +0000 (0:00:00.388) 0:03:26.899 ******* 2019-12-09T09:38:32.286 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T09:38:32.287 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T09:38:32.329 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:32 +0000 (0:00:00.243) 0:03:27.143 ******* 2019-12-09T09:38:32.445 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:38:32.446 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T09:38:32.488 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:32 +0000 (0:00:00.160) 0:03:27.304 ******* 2019-12-09T09:38:33.220 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-09T09:38:33.222 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T09:38:33.261 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:33 +0000 (0:00:00.771) 0:03:28.075 ******* 2019-12-09T09:38:33.425 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:38:33.426 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T09:38:33.462 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:33 +0000 (0:00:00.202) 0:03:28.277 ******* 2019-12-09T09:38:34.229 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T09:38:34.229 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T09:38:34.269 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:34 +0000 (0:00:00.806) 0:03:29.084 ******* 2019-12-09T09:38:34.506 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:38:34.507 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T09:38:34.552 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:34 +0000 (0:00:00.281) 0:03:29.365 ******* 2019-12-09T09:38:34.597 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:38:34.598 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:38:34.643 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:34 +0000 (0:00:00.092) 0:03:29.458 ******* 2019-12-09T09:38:34.969 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T09:38:35.031 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:35 +0000 (0:00:00.388) 0:03:29.847 ******* 2019-12-09T09:38:35.398 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:38:35.400 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T09:38:35.448 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:35 +0000 (0:00:00.412) 0:03:30.260 ******* 2019-12-09T09:38:36.395 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T09:38:36.435 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:36 +0000 (0:00:00.990) 0:03:31.251 ******* 2019-12-09T09:38:36.988 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:38:36.990 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.117284", "end": "2019-12-09 09:38:36.377970", "rc": 0, "start": "2019-12-09 09:38:34.260686", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:38:37.027 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:37 +0000 (0:00:00.592) 0:03:31.843 ******* 2019-12-09T09:38:37.055 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T09:38:37.113 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:37 +0000 (0:00:00.082) 0:03:31.926 ******* 2019-12-09T09:38:37.373 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T09:38:37.419 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:37 +0000 (0:00:00.306) 0:03:32.233 ******* 2019-12-09T09:38:38.120 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi161.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T09:38:39.055 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T09:38:39.108 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:39 +0000 (0:00:01.690) 0:03:33.923 ******* 2019-12-09T09:38:39.135 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T09:38:39.137 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:39.193 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:39 +0000 (0:00:00.085) 0:03:34.009 ******* 2019-12-09T09:38:39.244 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T09:38:39.245 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:38:39.256 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:38:39.306 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:39 +0000 (0:00:00.112) 0:03:34.122 ******* 2019-12-09T09:38:39.348 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T09:38:39.349 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:39.387 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:39 +0000 (0:00:00.081) 0:03:34.203 ******* 2019-12-09T09:38:39.424 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:38:39.426 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T09:38:39.463 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:39 +0000 (0:00:00.075) 0:03:34.278 ******* 2019-12-09T09:38:39.989 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T09:38:40.033 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:40 +0000 (0:00:00.564) 0:03:34.843 ******* 2019-12-09T09:38:40.081 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.335278", "end": "2019-12-09 09:38:40.045609", "rc": 0, "start": "2019-12-09 09:38:37.710331", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:38:40.122 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:40 +0000 (0:00:00.094) 0:03:34.937 ******* 2019-12-09T09:38:40.149 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T09:38:40.187 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:40 +0000 (0:00:00.064) 0:03:35.002 ******* 2019-12-09T09:38:40.618 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T09:38:40.625 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.263536", "end": "2019-12-09 09:38:40.003766", "rc": 0, "start": "2019-12-09 09:38:39.740230", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.ash.fastserv.com", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T09:38:40.673 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:40 +0000 (0:00:00.486) 0:03:35.489 ******* 2019-12-09T09:38:41.062 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T09:38:41.063 INFO:teuthology.task.ansible.out:fatal: [smithi161.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.056710", "end": "2019-12-09 09:38:40.451464", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:38:40.394754", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-09T09:38:41.104 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:41 +0000 (0:00:00.430) 0:03:35.920 ******* 2019-12-09T09:38:41.141 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T09:38:41.142 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:41.181 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:41 +0000 (0:00:00.076) 0:03:35.996 ******* 2019-12-09T09:38:41.186 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T09:38:41.187 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T09:38:42.142 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T09:38:42.217 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:42 +0000 (0:00:01.035) 0:03:37.032 ******* 2019-12-09T09:38:42.258 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T09:38:42.259 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:42.296 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:42 +0000 (0:00:00.080) 0:03:37.112 ******* 2019-12-09T09:38:42.334 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T09:38:42.336 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:38:42.346 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:38:42.390 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:42 +0000 (0:00:00.093) 0:03:37.206 ******* 2019-12-09T09:38:42.450 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T09:38:42.452 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:42.497 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:42 +0000 (0:00:00.101) 0:03:37.307 ******* 2019-12-09T09:38:42.532 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T09:38:42.591 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:42 +0000 (0:00:00.100) 0:03:37.407 ******* 2019-12-09T09:38:43.136 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-09T09:38:43.137 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T09:38:43.197 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:43 +0000 (0:00:00.604) 0:03:38.012 ******* 2019-12-09T09:38:43.880 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T09:38:43.881 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.331156", "end": "2019-12-09 09:38:43.837631", "rc": 0, "start": "2019-12-09 09:38:43.506475", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.fileplanet.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.fileplanet.com", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T09:38:43.927 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:43 +0000 (0:00:00.731) 0:03:38.743 ******* 2019-12-09T09:38:44.318 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T09:38:44.319 INFO:teuthology.task.ansible.out:fatal: [smithi059.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.052850", "end": "2019-12-09 09:38:44.288131", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:38:44.235281", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} 2019-12-09T09:38:44.320 INFO:teuthology.task.ansible.out:...ignoring 2019-12-09T09:38:44.362 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:44 +0000 (0:00:00.434) 0:03:39.178 ******* 2019-12-09T09:38:44.400 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:38:44.439 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:44 +0000 (0:00:00.076) 0:03:39.254 ******* 2019-12-09T09:38:49.623 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T09:38:49.625 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T09:38:49.626 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T09:38:49.628 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:38:49.665 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:49 +0000 (0:00:05.226) 0:03:44.481 ******* 2019-12-09T09:38:52.692 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T09:38:52.694 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T09:38:52.696 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:38:52.734 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:52 +0000 (0:00:03.069) 0:03:47.550 ******* 2019-12-09T09:38:53.019 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:38:53.020 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T09:38:53.075 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:53 +0000 (0:00:00.340) 0:03:47.891 ******* 2019-12-09T09:38:56.198 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T09:38:56.238 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:38:56 +0000 (0:00:03.162) 0:03:51.054 ******* 2019-12-09T09:39:11.299 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:39:11.306 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T09:39:11.307 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T09:39:11.309 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies 2019-12-09T09:39:11.310 INFO:teuthology.task.ansible.out:Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 26 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T09:39:11.312 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:39:11.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:11 +0000 (0:00:15.101) 0:04:06.155 ******* 2019-12-09T09:39:12.120 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-09T09:39:12.121 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T09:39:12.183 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:12 +0000 (0:00:00.842) 0:04:06.998 ******* 2019-12-09T09:39:12.712 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T09:39:12.752 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:12 +0000 (0:00:00.569) 0:04:07.567 ******* 2019-12-09T09:39:12.789 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:12.832 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:12 +0000 (0:00:00.080) 0:04:07.648 ******* 2019-12-09T09:39:13.277 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-09T09:39:13.288 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service system.slice basic.target dbus.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ip6tables.service ipset.service shutdown.target iptables.service ebtables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-09T09:39:13.289 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-09T09:39:13.291 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:39:13.331 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:13 +0000 (0:00:00.498) 0:04:08.147 ******* 2019-12-09T09:39:13.756 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi161.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T09:39:13.796 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:13 +0000 (0:00:00.465) 0:04:08.612 ******* 2019-12-09T09:39:14.661 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-09T09:39:14.663 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T09:39:14.707 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:14 +0000 (0:00:00.910) 0:04:09.523 ******* 2019-12-09T09:39:14.734 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T09:39:14.736 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:14.790 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:14 +0000 (0:00:00.083) 0:04:09.606 ******* 2019-12-09T09:39:14.816 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T09:39:14.853 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:14 +0000 (0:00:00.063) 0:04:09.669 ******* 2019-12-09T09:39:14.881 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-09T09:39:14.883 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:14.919 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:14 +0000 (0:00:00.065) 0:04:09.734 ******* 2019-12-09T09:39:14.945 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T09:39:14.947 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:14.984 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:14 +0000 (0:00:00.065) 0:04:09.800 ******* 2019-12-09T09:39:15.024 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* skipping: [smithi161.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.065 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.081) 0:04:09.881 ******* 2019-12-09T09:39:15.093 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-09T09:39:15.094 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.149 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.083) 0:04:09.965 ******* 2019-12-09T09:39:15.151 INFO:teuthology.task.ansible.out: 2019-12-09T09:39:15.189 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T09:39:15.248 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.099) 0:04:10.064 ******* 2019-12-09T09:39:15.278 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.314 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.065) 0:04:10.129 ******* 2019-12-09T09:39:15.340 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T09:39:15.342 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.390 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.075) 0:04:10.205 ******* 2019-12-09T09:39:15.396 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:39:15.399 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T09:39:15.400 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T09:39:15.402 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.fileplanet.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Re 2019-12-09T09:39:15.403 INFO:teuthology.task.ansible.out:solved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 23 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T09:39:15.405 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:39:15.468 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.078) 0:04:10.284 ******* 2019-12-09T09:39:15.480 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T09:39:15.481 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.519 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.050) 0:04:10.335 ******* 2019-12-09T09:39:15.546 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:39:15.547 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.587 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.067) 0:04:10.402 ******* 2019-12-09T09:39:15.624 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.669 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.082) 0:04:10.485 ******* 2019-12-09T09:39:15.708 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:39:15.711 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.758 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.089) 0:04:10.574 ******* 2019-12-09T09:39:15.785 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T09:39:15.786 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.830 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.071) 0:04:10.645 ******* 2019-12-09T09:39:15.868 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-09T09:39:15.870 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.907 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.077) 0:04:10.723 ******* 2019-12-09T09:39:15.939 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-09T09:39:15.941 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:15.977 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:15 +0000 (0:00:00.069) 0:04:10.793 ******* 2019-12-09T09:39:16.006 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.044 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.066) 0:04:10.860 ******* 2019-12-09T09:39:16.071 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.129 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.064) 0:04:10.924 ******* 2019-12-09T09:39:16.135 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-09T09:39:16.137 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.176 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.066) 0:04:10.991 ******* 2019-12-09T09:39:16.203 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T09:39:16.205 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.241 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.065) 0:04:11.057 ******* 2019-12-09T09:39:16.268 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T09:39:16.269 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.316 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.070) 0:04:11.127 ******* 2019-12-09T09:39:16.318 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T09:39:16.357 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.045) 0:04:11.173 ******* 2019-12-09T09:39:16.363 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:39:16.364 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.046) 0:04:11.219 ******* 2019-12-09T09:39:16.431 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:39:16.433 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.476 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.072) 0:04:11.291 ******* 2019-12-09T09:39:16.503 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.552 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.076) 0:04:11.368 ******* 2019-12-09T09:39:16.579 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:39:16.582 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.617 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.064) 0:04:11.432 ******* 2019-12-09T09:39:16.649 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.687 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.070) 0:04:11.503 ******* 2019-12-09T09:39:16.714 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:39:16.716 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.755 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.066) 0:04:11.569 ******* 2019-12-09T09:39:16.790 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T09:39:16.796 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.828 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.074) 0:04:11.643 ******* 2019-12-09T09:39:16.854 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:39:16.855 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:16.902 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.074) 0:04:11.718 ******* 2019-12-09T09:39:16.907 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:39:16.908 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T09:39:16.947 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.044) 0:04:11.763 ******* 2019-12-09T09:39:16.953 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:39:16.955 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:17.001 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:16 +0000 (0:00:00.049) 0:04:11.812 ******* 2019-12-09T09:39:17.002 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:39:17.004 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:17.042 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:17 +0000 (0:00:00.045) 0:04:11.858 ******* 2019-12-09T09:39:17.049 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:39:17.050 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:17.089 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:17 +0000 (0:00:00.046) 0:04:11.904 ******* 2019-12-09T09:39:17.126 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:39:17.128 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:17.176 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:17 +0000 (0:00:00.087) 0:04:11.992 ******* 2019-12-09T09:39:17.206 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:39:17.216 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:17.249 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:17 +0000 (0:00:00.073) 0:04:12.065 ******* 2019-12-09T09:39:17.525 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-09T09:39:17.527 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service polkit.service basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "iptables.service ip6tables.service ebtables.service ipset.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-09T09:39:17.529 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-09T09:39:17.531 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:39:17.570 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:17 +0000 (0:00:00.320) 0:04:12.385 ******* 2019-12-09T09:39:17.589 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T09:39:17.626 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:17 +0000 (0:00:00.056) 0:04:12.442 ******* 2019-12-09T09:39:17.997 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-09T09:39:17.999 INFO:teuthology.task.ansible.out:fatal: [smithi059.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T09:39:18.048 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.421) 0:04:12.864 ******* 2019-12-09T09:39:18.055 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:39:18.056 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:39:18.101 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.052) 0:04:12.917 ******* 2019-12-09T09:39:18.458 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:39:18.459 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T09:39:18.503 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.400) 0:04:13.317 ******* 2019-12-09T09:39:18.512 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T09:39:18.549 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.047) 0:04:13.365 ******* 2019-12-09T09:39:18.555 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:39:18.555 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.598 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.048) 0:04:13.413 ******* 2019-12-09T09:39:18.607 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T09:39:18.608 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.644 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.046) 0:04:13.460 ******* 2019-12-09T09:39:18.653 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.692 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.047) 0:04:13.508 ******* 2019-12-09T09:39:18.697 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T09:39:18.746 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.049) 0:04:13.557 ******* 2019-12-09T09:39:18.748 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:39:18.750 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.791 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.048) 0:04:13.606 ******* 2019-12-09T09:39:18.798 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-09T09:39:18.799 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.843 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.052) 0:04:13.659 ******* 2019-12-09T09:39:18.850 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:39:18.851 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.888 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.044) 0:04:13.704 ******* 2019-12-09T09:39:18.893 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T09:39:18.895 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.944 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.048) 0:04:13.752 ******* TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.984 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:18 +0000 (0:00:00.047) 0:04:13.800 ******* 2019-12-09T09:39:18.989 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-09T09:39:18.990 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:18.996 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.032 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.047) 0:04:13.847 ******* 2019-12-09T09:39:19.038 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:39:19.039 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.090 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.058) 0:04:13.906 ******* 2019-12-09T09:39:19.097 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-09T09:39:19.099 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.138 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.047) 0:04:13.954 ******* 2019-12-09T09:39:19.147 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:39:19.149 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.192 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.054) 0:04:14.008 ******* 2019-12-09T09:39:19.199 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T09:39:19.253 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.060) 0:04:14.069 ******* 2019-12-09T09:39:19.262 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:39:19.264 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.304 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.050) 0:04:14.119 ******* 2019-12-09T09:39:19.310 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-09T09:39:19.310 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.352 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.048) 0:04:14.168 ******* 2019-12-09T09:39:19.364 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.417 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.064) 0:04:14.232 ******* 2019-12-09T09:39:19.423 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T09:39:19.425 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.478 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.061) 0:04:14.294 ******* 2019-12-09T09:39:19.485 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.523 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.044) 0:04:14.338 ******* 2019-12-09T09:39:19.529 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T09:39:19.530 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.567 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.044) 0:04:14.383 ******* 2019-12-09T09:39:19.573 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-09T09:39:19.580 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.613 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.045) 0:04:14.428 ******* 2019-12-09T09:39:19.620 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:39:19.622 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.658 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.045) 0:04:14.474 ******* 2019-12-09T09:39:19.665 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:39:19.667 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.705 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.046) 0:04:14.520 ******* 2019-12-09T09:39:19.710 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-09T09:39:19.712 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.765 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.059) 0:04:14.580 ******* 2019-12-09T09:39:19.773 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:39:19.775 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.815 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.051) 0:04:14.631 ******* 2019-12-09T09:39:19.821 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:39:19.821 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.861 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.045) 0:04:14.677 ******* 2019-12-09T09:39:19.867 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T09:39:19.869 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.043) 0:04:14.721 ******* 2019-12-09T09:39:19.910 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T09:39:19.912 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.948 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.043) 0:04:14.764 ******* 2019-12-09T09:39:19.957 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-09T09:39:19.959 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:19.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:19 +0000 (0:00:00.045) 0:04:14.810 ******* 2019-12-09T09:39:20.004 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.038 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.044) 0:04:14.854 ******* 2019-12-09T09:39:20.048 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.084 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.045) 0:04:14.900 ******* 2019-12-09T09:39:20.089 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-09T09:39:20.091 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.128 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.044) 0:04:14.944 ******* 2019-12-09T09:39:20.133 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T09:39:20.135 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.175 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.046) 0:04:14.991 ******* 2019-12-09T09:39:20.182 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T09:39:20.184 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.223 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.047) 0:04:15.038 ******* 2019-12-09T09:39:20.228 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T09:39:20.230 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.280 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.057) 0:04:15.096 ******* 2019-12-09T09:39:20.288 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T09:39:20.289 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.338 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.057) 0:04:15.154 ******* 2019-12-09T09:39:20.344 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-09T09:39:20.345 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.382 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.043) 0:04:15.197 ******* 2019-12-09T09:39:20.389 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.428 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.046) 0:04:15.244 ******* 2019-12-09T09:39:20.434 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-09T09:39:20.436 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.472 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.044) 0:04:15.288 ******* 2019-12-09T09:39:20.478 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T09:39:20.479 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.517 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.044) 0:04:15.332 ******* 2019-12-09T09:39:20.523 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-09T09:39:20.525 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.562 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.045) 0:04:15.378 ******* 2019-12-09T09:39:20.567 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T09:39:20.569 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.606 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.043) 0:04:15.421 ******* 2019-12-09T09:39:20.614 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T09:39:20.616 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.658 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.052) 0:04:15.474 ******* 2019-12-09T09:39:20.664 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:39:20.666 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.707 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.048) 0:04:15.522 ******* 2019-12-09T09:39:20.715 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T09:39:20.762 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.055) 0:04:15.578 ******* 2019-12-09T09:39:20.769 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.811 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.048) 0:04:15.626 ******* 2019-12-09T09:39:20.817 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:39:20.819 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.830 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T09:39:20.841 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:39:20.851 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.882 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.071) 0:04:15.698 ******* 2019-12-09T09:39:20.896 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi161.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.934 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.051) 0:04:15.749 ******* 2019-12-09T09:39:20.939 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:39:20.941 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:20.978 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:20 +0000 (0:00:00.044) 0:04:15.794 ******* 2019-12-09T09:39:21.016 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:39:21.017 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.058 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.079) 0:04:15.873 ******* 2019-12-09T09:39:21.086 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:39:21.088 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.127 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.068) 0:04:15.942 ******* 2019-12-09T09:39:21.134 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T09:39:21.136 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.006198", "end": "2019-12-09 09:39:21.072193", "rc": 0, "start": "2019-12-09 09:39:21.065995", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T09:39:21.192 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.064) 0:04:16.007 ******* 2019-12-09T09:39:21.198 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.252 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.061) 0:04:16.068 ******* 2019-12-09T09:39:21.258 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T09:39:21.259 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.304 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.051) 0:04:16.119 ******* 2019-12-09T09:39:21.309 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:39:21.312 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.353 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.049) 0:04:16.169 ******* 2019-12-09T09:39:21.360 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-09T09:39:21.362 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.049) 0:04:16.219 ******* 2019-12-09T09:39:21.409 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.451 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.047) 0:04:16.267 ******* 2019-12-09T09:39:21.482 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.499 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.047) 0:04:16.315 ******* 2019-12-09T09:39:21.500 INFO:teuthology.task.ansible.out: 2019-12-09T09:39:21.505 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:39:21.505 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.545 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.046) 0:04:16.361 ******* 2019-12-09T09:39:21.553 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.592 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.047) 0:04:16.408 ******* 2019-12-09T09:39:21.601 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:39:21.602 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.647 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.054) 0:04:16.462 ******* 2019-12-09T09:39:21.674 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:39:21.675 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:21.714 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:21 +0000 (0:00:00.067) 0:04:16.529 ******* 2019-12-09T09:39:22.053 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:39:22.054 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T09:39:22.097 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.382) 0:04:16.912 ******* 2019-12-09T09:39:22.102 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T09:39:22.103 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T09:39:22.143 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.046) 0:04:16.959 ******* 2019-12-09T09:39:22.181 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:39:22.182 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item={u'block_used': 13040463, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284021, u'size_available': 930955350016, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284021, "block_size": 4096, "block_total": 240324484, "block_used": 13040463, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955350016, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T09:39:22.227 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.083) 0:04:17.043 ******* 2019-12-09T09:39:22.435 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:39:22.474 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.247) 0:04:17.290 ******* 2019-12-09T09:39:22.568 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T09:39:22.666 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.012324", "end": "2019-12-09 09:39:21.965379", "rc": 0, "start": "2019-12-09 09:39:21.953055", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Monday 09 December 2019 09:39:22 +0000 (0:00:00.148) 0:04:17.439 ******* TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi161.front.sepia.ceph.com] => (item={u'block_used': 13040463, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284021, u'size_available': 930955350016, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284021, "block_size": 4096, "block_total": 240324484, "block_used": 13040463, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955350016, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:22.701 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.078) 0:04:17.517 ******* 2019-12-09T09:39:22.728 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T09:39:22.766 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.064) 0:04:17.581 ******* 2019-12-09T09:39:22.808 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XV0E', u'wwn-0x5000c50092938c4f'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XV0E', u'wwn': u'0x5000c50092938c4f', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XV0E-part1', u'wwn-0x5000c50092938c4f-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XV0E", "wwn-0x5000c50092938c4f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XV0E-part1", "wwn-0x5000c50092938c4f-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": " 2019-12-09T09:39:22.810 INFO:teuthology.task.ansible.out:512", "serial": "Z1W5XV0E", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092938c4f"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:22.826 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:39:22.827 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T09:39:22.879 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.113) 0:04:17.694 ******* 2019-12-09T09:39:22.919 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:22.955 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:22 +0000 (0:00:00.076) 0:04:17.771 ******* 2019-12-09T09:39:22.985 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:39:22.986 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.043 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.087) 0:04:17.858 ******* 2019-12-09T09:39:23.072 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:39:23.073 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.121 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.078) 0:04:17.936 ******* 2019-12-09T09:39:23.148 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:39:23.149 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.197 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.076) 0:04:18.013 ******* 2019-12-09T09:39:23.227 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:39:23.228 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.264 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.066) 0:04:18.080 ******* 2019-12-09T09:39:23.291 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.328 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.063) 0:04:18.143 ******* 2019-12-09T09:39:23.366 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:39:23.368 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.076) 0:04:18.220 ******* 2019-12-09T09:39:23.442 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.481 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.077) 0:04:18.297 ******* 2019-12-09T09:39:23.544 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:39:23.546 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.580 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.098) 0:04:18.395 ******* 2019-12-09T09:39:23.581 INFO:teuthology.task.ansible.out: 2019-12-09T09:39:23.619 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.659 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.077) 0:04:18.473 ******* 2019-12-09T09:39:23.688 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-09T09:39:23.690 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.741 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.083) 0:04:18.557 ******* 2019-12-09T09:39:23.768 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:39:23.770 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.825 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.083) 0:04:18.640 ******* 2019-12-09T09:39:23.851 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:39:23.853 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.893 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.067) 0:04:18.708 ******* 2019-12-09T09:39:23.921 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:23.960 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:23 +0000 (0:00:00.066) 0:04:18.775 ******* 2019-12-09T09:39:23.996 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-09T09:39:23.998 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.038 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.076) 0:04:18.851 ******* 2019-12-09T09:39:24.062 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-09T09:39:24.064 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.103 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.067) 0:04:18.919 ******* 2019-12-09T09:39:24.130 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T09:39:24.131 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.169 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.065) 0:04:18.984 ******* 2019-12-09T09:39:24.196 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T09:39:24.198 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.235 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.065) 0:04:19.050 ******* 2019-12-09T09:39:24.271 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.309 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.074) 0:04:19.125 ******* 2019-12-09T09:39:24.336 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-09T09:39:24.337 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.402 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.093) 0:04:19.218 ******* 2019-12-09T09:39:24.426 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-09T09:39:24.428 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.463 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.061) 0:04:19.279 ******* 2019-12-09T09:39:24.491 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T09:39:24.492 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.540 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.074) 0:04:19.353 ******* 2019-12-09T09:39:24.565 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:39:24.567 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T09:39:24.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.076) 0:04:19.430 ******* 2019-12-09T09:39:24.663 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi059.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.674 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T09:39:24.685 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.686 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.697 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.700 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:39:24.701 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-kV2yun-Ew8S-j4UB-ZEQj-h9o1-VToj-qmxFuc', u'nvme-INTEL_SSDPEDMD400G4_CVFT6233001X400BGN', u'nvme-nvme.8086-43564654363233333030315834303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.598808", "end": "2019-12-09 09:39:24.090412", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-kV2yun-Ew8S-j4UB-ZEQj-h9o1-VToj-qmxFuc", "nvme-INTEL_SSDPEDMD400G4_CVFT6233001X400BGN", "nvme-nvme.8086-43564654363233333030315834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:22.491 2019-12-09T09:39:24.702 INFO:teuthology.task.ansible.out:604", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:24.732 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:39:24.733 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:24.773 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:24 +0000 (0:00:00.158) 0:04:19.589 ******* 2019-12-09T09:39:24.931 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T09:39:24.932 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004131", "end": "2019-12-09 09:39:24.900915", "rc": 0, "start": "2019-12-09 09:39:24.896784", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T09:39:25.000 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.227) 0:04:19.816 ******* 2019-12-09T09:39:25.028 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T09:39:25.029 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:25.070 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.069) 0:04:19.886 ******* 2019-12-09T09:39:25.099 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-09T09:39:25.100 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:25.138 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.067) 0:04:19.953 ******* 2019-12-09T09:39:25.175 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T09:39:25.177 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:25.213 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.075) 0:04:20.028 ******* 2019-12-09T09:39:25.249 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-09T09:39:25.251 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:25.286 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.073) 0:04:20.102 ******* 2019-12-09T09:39:25.789 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T09:39:25.837 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.550) 0:04:20.653 ******* 2019-12-09T09:39:25.885 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:39:25.886 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T09:39:25.924 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:25 +0000 (0:00:00.087) 0:04:20.740 ******* 2019-12-09T09:39:26.076 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:39:26.077 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LS1ZoLRXBYJZH871shtbcu9t3rFynbffH'], u'uuids': [u'6452be34-4aa4-49d7-8175-ff81059e43c0']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.071716", "end": "2019-12-09 09:39:25.467566", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LS1ZoLRXBYJZH871shtbcu9t3rFynbffH"], "labels": [], "masters": [], "uuids": ["6452be34-4aa4-49d7-8175-ff81059e43c0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:24.395850", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:26.262 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T09:39:26.262 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013593", "end": "2019-12-09 09:39:26.233051", "rc": 0, "start": "2019-12-09 09:39:26.219458", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:26.306 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:26 +0000 (0:00:00.382) 0:04:21.122 ******* 2019-12-09T09:39:26.357 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-09T09:39:26.358 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:26.406 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:26 +0000 (0:00:00.099) 0:04:21.222 ******* 2019-12-09T09:39:26.433 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T09:39:26.485 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:26 +0000 (0:00:00.078) 0:04:21.301 ******* 2019-12-09T09:39:26.533 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:39:26.534 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2P7', u'wwn-0x5000c500871b7b3d'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4J2P7', u'wwn': u'0x5000c500871b7b3d', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2P7-part1', u'wwn-0x5000c500871b7b3d-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2P7", "wwn-0x5000c500871b7b3d"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2P7-part1", "wwn-0x5000c500871b7b3d-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2P7", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:39:26.536 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871b7b3d"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:27.464 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LZgYWdn9BkujWY3CaDxqJYGS6EbFDwKmI'], u'uuids': [u'2a187bfc-e64e-46f8-a998-e3369139fe86']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.065478", "end": "2019-12-09 09:39:26.827323", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LZgYWdn9BkujWY3CaDxqJYGS6EbFDwKmI"], "labels": [], "masters": [], "uuids": ["2a187bfc-e64e-46f8-a998-e3369139fe86"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:25.761845", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:28.326 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-MChLPd-22Rb-rkfq-dKr2-WyEp-o8Z8-BfCjr8', u'nvme-INTEL_SSDPEDMD400G4_CVFT5330009L400BGN', u'nvme-nvme.8086-43564654353333303030394c34303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.461760", "end": "2019-12-09 09:39:28.281546", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-MChLPd-22Rb-rkfq-dKr2-WyEp-o8Z8-BfCjr8", "nvme-INTEL_SSDPEDMD400G4_CVFT5330009L400BGN", "nvme-nvme.8086-43564654353333303030394c34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:26.819 2019-12-09T09:39:28.328 INFO:teuthology.task.ansible.out:786", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:28.824 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LFpoLzLTcQdUcNy1K9WhcfeoINoDPFgMV'], u'uuids': [u'96854edd-98b6-4eca-b722-2ebeff142e5d']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.057536", "end": "2019-12-09 09:39:28.201153", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LFpoLzLTcQdUcNy1K9WhcfeoINoDPFgMV"], "labels": [], "masters": [], "uuids": ["96854edd-98b6-4eca-b722-2ebeff142e5d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:27.143617", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:29.693 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157OSXVkVgnq2EljKvrGk2SbkoCTB4rxAbQ'], u'uuids': [u'c96454a5-870c-46fc-8b5d-6479ed440d60']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.049025", "end": "2019-12-09 09:39:29.656548", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157OSXVkVgnq2EljKvrGk2SbkoCTB4rxAbQ"], "labels": [], "masters": [], "uuids": ["c96454a5-870c-46fc-8b5d-6479ed440d60"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:28.607523", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:30.193 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945L5zSuM82Vzjg2ULhTlC5BytMPlr28hIM2'], u'uuids': [u'103ac2ad-9f71-4e97-9733-ef3bf0259d52']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.063836", "end": "2019-12-09 09:39:29.581058", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945L5zSuM82Vzjg2ULhTlC5BytMPlr28hIM2"], "labels": [], "masters": [], "uuids": ["103ac2ad-9f71-4e97-9733-ef3bf0259d52"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:28.517222", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:31.073 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157uRrLfZ5FKfJlfEgboD6mXLbku8CoxRpL'], u'uuids': [u'9d106741-5053-4227-ac68-a21951e6f79f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.058915", "end": "2019-12-09 09:39:31.032438", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157uRrLfZ5FKfJlfEgboD6mXLbku8CoxRpL"], "labels": [], "masters": [], "uuids": ["9d106741-5053-4227-ac68-a21951e6f79f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:29.973523", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:31.556 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LcncnAHFl2ep5dq2ukEwU2CyMLgle7puq'], u'uuids': [u'7c4ca6b4-d30a-4376-92a3-33bee3c958e7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.060886", "end": "2019-12-09 09:39:30.943694", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LcncnAHFl2ep5dq2ukEwU2CyMLgle7puq"], "labels": [], "masters": [], "uuids": ["7c4ca6b4-d30a-4376-92a3-33bee3c958e7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:29.882808", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:31.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:31 +0000 (0:00:05.128) 0:04:26.429 ******* 2019-12-09T09:39:31.677 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:31.678 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XV0E', u'wwn-0x5000c50092938c4f'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XV0E', u'wwn': u'0x5000c50092938c4f', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XV0E-part1', u'wwn-0x5000c50092938c4f-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XV0E", "wwn-0x5000c50092938c4f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XV0E-part1", "wwn-0x5000c50092938c4f-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XV0E", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:39:31.679 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50092938c4f"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:32.044 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-kV2yun-Ew8S-j4UB-ZEQj-h9o1-VToj-qmxFuc', u'nvme-INTEL_SSDPEDMD400G4_CVFT6233001X400BGN', u'nvme-nvme.8086-43564654363233333030315834303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.067675", "end": "2019-12-09 09:39:31.429461", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-kV2yun-Ew8S-j4UB-ZEQj-h9o1-VToj-qmxFuc", "nvme-INTEL_SSDPEDMD400G4_CVFT6233001X400BGN", "nvme-nvme.8086-43564654363233333030315834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:31.361786", "stderr": 2019-12-09T09:39:32.046 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0615744 s, 1.9 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0615744 s, 1.9 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:32.460 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:39:32.461 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157E3EOjuCFyfb7eEejuYR8AXed4ZKZOf3p'], u'uuids': [u'bc30d302-e192-46d4-96c1-3d4b48670f92']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.078900", "end": "2019-12-09 09:39:32.424447", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157E3EOjuCFyfb7eEejuYR8AXed4ZKZOf3p"], "labels": [], "masters": [], "uuids": ["bc30d302-e192-46d4-96c1-3d4b48670f92"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:31.345547", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:32.473 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:32.474 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LS1ZoLRXBYJZH871shtbcu9t3rFynbffH'], u'uuids': [u'6452be34-4aa4-49d7-8175-ff81059e43c0']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.162416", "end": "2019-12-09 09:39:31.867004", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LS1ZoLRXBYJZH871shtbcu9t3rFynbffH"], "labels": [], "masters": [], "uuids": ["6452be34-4aa4-49d7-8175-ff81059e43c0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:31.704588", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.156933 s, 735 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.156933 s, 735 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:32.999 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LZgYWdn9BkujWY3CaDxqJYGS6EbFDwKmI'], u'uuids': [u'2a187bfc-e64e-46f8-a998-e3369139fe86']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.213018", "end": "2019-12-09 09:39:32.382940", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LZgYWdn9BkujWY3CaDxqJYGS6EbFDwKmI"], "labels": [], "masters": [], "uuids": ["2a187bfc-e64e-46f8-a998-e3369139fe86"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:32.169922", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.206287 s, 559 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.206287 s, 559 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:33.506 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LFpoLzLTcQdUcNy1K9WhcfeoINoDPFgMV'], u'uuids': [u'96854edd-98b6-4eca-b722-2ebeff142e5d']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.173071", "end": "2019-12-09 09:39:32.897862", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LFpoLzLTcQdUcNy1K9WhcfeoINoDPFgMV"], "labels": [], "masters": [], "uuids": ["96854edd-98b6-4eca-b722-2ebeff142e5d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:32.724791", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.167322 s, 689 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.167322 s, 689 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:33.850 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:39:33.851 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157atoecb0EhY1C43902j4Gs2pO1MlUoF9W'], u'uuids': [u'5c04f808-619f-418b-b317-641dce7a892c']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.062630", "end": "2019-12-09 09:39:33.808774", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157atoecb0EhY1C43902j4Gs2pO1MlUoF9W"], "labels": [], "masters": [], "uuids": ["5c04f808-619f-418b-b317-641dce7a892c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:32.746144", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:34.027 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:34.028 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945L5zSuM82Vzjg2ULhTlC5BytMPlr28hIM2'], u'uuids': [u'103ac2ad-9f71-4e97-9733-ef3bf0259d52']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.196616", "end": "2019-12-09 09:39:33.418442", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945L5zSuM82Vzjg2ULhTlC5BytMPlr28hIM2"], "labels": [], "masters": [], "uuids": ["103ac2ad-9f71-4e97-9733-ef3bf0259d52"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:33.221826", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.189851 s, 608 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.189851 s, 608 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:34.494 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LcncnAHFl2ep5dq2ukEwU2CyMLgle7puq'], u'uuids': [u'7c4ca6b4-d30a-4376-92a3-33bee3c958e7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6233001X400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.159670", "end": "2019-12-09 09:39:33.880103", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-0Id05VtS3mS1XGwyTuuRBzdVgieD945LcncnAHFl2ep5dq2ukEwU2CyMLgle7puq"], "labels": [], "masters": [], "uuids": ["7c4ca6b4-d30a-4376-92a3-33bee3c958e7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6233001X400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:33.720433", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.153324 s, 752 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.153324 s, 752 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:34.540 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:34 +0000 (0:00:02.926) 0:04:29.356 ******* 2019-12-09T09:39:34.971 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-09T09:39:34.973 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.086395", "end": "2019-12-09 09:39:34.357187", "rc": 0, "start": "2019-12-09 09:39:34.270792", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:35.014 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.473) 0:04:29.830 ******* 2019-12-09T09:39:35.275 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:39:35.276 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157ijZg1iQB3VGASfp0r3IiCJjvSXFkXV0p'], u'uuids': [u'8661bb67-08d8-40d9-82ae-fbc79de48d2a']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.101801", "end": "2019-12-09 09:39:35.235922", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157ijZg1iQB3VGASfp0r3IiCJjvSXFkXV0p"], "labels": [], "masters": [], "uuids": ["8661bb67-08d8-40d9-82ae-fbc79de48d2a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:34.134121", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:39:35.344 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.316) 0:04:30.147 ******* 2019-12-09T09:39:35.381 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:35.382 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2P7', u'wwn-0x5000c500871b7b3d'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4J2P7', u'wwn': u'0x5000c500871b7b3d', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J2P7-part1', u'wwn-0x5000c500871b7b3d-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2P7", "wwn-0x5000c500871b7b3d"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2P7-part1", "wwn-0x5000c500871b7b3d-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2P7", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:39:35.383 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871b7b3d"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:35.409 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T09:39:35.411 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.052290", "end": "2019-12-09 09:39:34.797405", "rc": 0, "start": "2019-12-09 09:39:34.745115", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:35.458 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.126) 0:04:30.273 ******* 2019-12-09T09:39:35.496 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T09:39:35.539 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.081) 0:04:30.355 ******* 2019-12-09T09:39:35.570 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-09T09:39:35.571 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:35.610 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.071) 0:04:30.426 ******* 2019-12-09T09:39:35.646 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:35.682 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.072) 0:04:30.498 ******* 2019-12-09T09:39:35.721 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:39:35.723 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:35.770 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.086) 0:04:30.584 ******* 2019-12-09T09:39:35.777 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-MChLPd-22Rb-rkfq-dKr2-WyEp-o8Z8-BfCjr8', u'nvme-INTEL_SSDPEDMD400G4_CVFT5330009L400BGN', u'nvme-nvme.8086-43564654353333303030394c34303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.062574", "end": "2019-12-09 09:39:35.716654", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-MChLPd-22Rb-rkfq-dKr2-WyEp-o8Z8-BfCjr8", "nvme-INTEL_SSDPEDMD400G4_CVFT5330009L400BGN", "nvme-nvme.8086-43564654353333303030394c34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "372.61 GB", "support_discard": "512", "vend 2019-12-09T09:39:35.778 INFO:teuthology.task.ansible.out:or": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:35.654080", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0561295 s, 2.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0561295 s, 2.1 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:35.798 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-09T09:39:35.800 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:35.840 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:35 +0000 (0:00:00.071) 0:04:30.656 ******* 2019-12-09T09:39:36.172 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:39:36.237 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.392) 0:04:31.049 ******* 2019-12-09T09:39:36.275 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:36.277 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157OSXVkVgnq2EljKvrGk2SbkoCTB4rxAbQ'], u'uuids': [u'c96454a5-870c-46fc-8b5d-6479ed440d60']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.198685", "end": "2019-12-09 09:39:36.231959", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157OSXVkVgnq2EljKvrGk2SbkoCTB4rxAbQ"], "labels": [], "masters": [], "uuids": ["c96454a5-870c-46fc-8b5d-6479ed440d60"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:36.033274", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.192124 s, 600 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.192124 s, 600 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:36.287 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:39:36.290 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={u'block_used': 13040463, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284021, u'size_available': 930955350016, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284021, "block_size": 4096, "block_total": 240324484, "block_used": 13040463, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955350016, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.327 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.094) 0:04:31.143 ******* 2019-12-09T09:39:36.357 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-09T09:39:36.359 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.403 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.075) 0:04:31.219 ******* 2019-12-09T09:39:36.444 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.488 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.084) 0:04:31.303 ******* 2019-12-09T09:39:36.530 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.076) 0:04:31.380 ******* 2019-12-09T09:39:36.601 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T09:39:36.602 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.643 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.074) 0:04:31.454 ******* 2019-12-09T09:39:36.682 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T09:39:36.684 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.722 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.080) 0:04:31.534 ******* 2019-12-09T09:39:36.747 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T09:39:36.749 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:36.796 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:36 +0000 (0:00:00.074) 0:04:31.609 ******* 2019-12-09T09:39:36.800 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:36.802 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157uRrLfZ5FKfJlfEgboD6mXLbku8CoxRpL'], u'uuids': [u'9d106741-5053-4227-ac68-a21951e6f79f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.177832", "end": "2019-12-09 09:39:36.728425", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157uRrLfZ5FKfJlfEgboD6mXLbku8CoxRpL"], "labels": [], "masters": [], "uuids": ["9d106741-5053-4227-ac68-a21951e6f79f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:36.550593", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.171029 s, 674 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.171029 s, 674 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:37.287 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157E3EOjuCFyfb7eEejuYR8AXed4ZKZOf3p'], u'uuids': [u'bc30d302-e192-46d4-96c1-3d4b48670f92']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.175863", "end": "2019-12-09 09:39:37.259153", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157E3EOjuCFyfb7eEejuYR8AXed4ZKZOf3p"], "labels": [], "masters": [], "uuids": ["bc30d302-e192-46d4-96c1-3d4b48670f92"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:37.083290", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.169739 s, 680 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.169739 s, 680 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:37.627 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T09:39:37.664 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:37 +0000 (0:00:00.870) 0:04:32.480 ******* 2019-12-09T09:39:37.829 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:39:37.830 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157atoecb0EhY1C43902j4Gs2pO1MlUoF9W'], u'uuids': [u'5c04f808-619f-418b-b317-641dce7a892c']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.196926", "end": "2019-12-09 09:39:37.774163", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157atoecb0EhY1C43902j4Gs2pO1MlUoF9W"], "labels": [], "masters": [], "uuids": ["5c04f808-619f-418b-b317-641dce7a892c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:37.577237", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.190563 s, 605 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.190563 s, 605 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:38.353 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157ijZg1iQB3VGASfp0r3IiCJjvSXFkXV0p'], u'uuids': [u'8661bb67-08d8-40d9-82ae-fbc79de48d2a']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330009L400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.195523", "end": "2019-12-09 09:39:38.314010", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-eLkvfLyChAmS4qfBUUP7dHelztSFt157ijZg1iQB3VGASfp0r3IiCJjvSXFkXV0p"], "labels": [], "masters": [], "uuids": ["8661bb67-08d8-40d9-82ae-fbc79de48d2a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330009L400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:39:38.118487", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.188625 s, 611 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.188625 s, 611 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:38.402 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:38 +0000 (0:00:00.738) 0:04:33.218 ******* 2019-12-09T09:39:38.483 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:39:38.484 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:38.826 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi059.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.079298", "end": "2019-12-09 09:39:38.780227", "rc": 0, "start": "2019-12-09 09:39:38.700929", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:38.857 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:38 +0000 (0:00:00.454) 0:04:33.673 ******* 2019-12-09T09:39:38.969 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:39:38.970 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:39.258 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T09:39:39.260 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.062688", "end": "2019-12-09 09:39:39.226487", "rc": 0, "start": "2019-12-09 09:39:39.163799", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:39.297 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:39 +0000 (0:00:00.440) 0:04:34.113 ******* 2019-12-09T09:39:39.325 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T09:39:39.366 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:39 +0000 (0:00:00.068) 0:04:34.181 ******* 2019-12-09T09:39:39.400 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:39.479 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:39 +0000 (0:00:00.113) 0:04:34.295 ******* 2019-12-09T09:39:39.490 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:39:39.491 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:39.515 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-09T09:39:39.516 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:39.554 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:39 +0000 (0:00:00.075) 0:04:34.370 ******* 2019-12-09T09:39:39.581 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:39:39.584 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:39.628 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:39 +0000 (0:00:00.070) 0:04:34.440 ******* 2019-12-09T09:39:39.652 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-09T09:39:39.653 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:39.690 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:39 +0000 (0:00:00.065) 0:04:34.506 ******* 2019-12-09T09:39:39.960 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:39:39.962 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:40.016 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-09T09:39:40.017 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:39:40.073 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.382) 0:04:34.888 ******* 2019-12-09T09:39:40.110 INFO:teuthology.task.ansible.out: 2019-12-09T09:39:40.112 INFO:teuthology.task.ansible.out:TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:39:40.113 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={u'block_used': 13040464, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284020, u'size_available': 930955345920, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284020, "block_size": 4096, "block_total": 240324484, "block_used": 13040464, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955345920, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:40.153 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.080) 0:04:34.969 ******* 2019-12-09T09:39:40.201 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-09T09:39:40.202 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:40.239 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.086) 0:04:35.055 ******* 2019-12-09T09:39:40.277 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:39:40.278 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:40.320 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.080) 0:04:35.135 ******* 2019-12-09T09:39:40.346 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:39:40.348 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:40.385 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.064) 0:04:35.200 ******* 2019-12-09T09:39:40.422 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T09:39:40.424 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:40.484 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.095) 0:04:35.296 ******* 2019-12-09T09:39:40.522 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.042) 0:04:35.338 ******* 2019-12-09T09:39:40.529 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T09:39:40.530 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:40.575 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.053) 0:04:35.391 ******* 2019-12-09T09:39:40.635 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:40.681 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.105) 0:04:35.497 ******* 2019-12-09T09:39:40.841 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T09:39:40.881 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:40 +0000 (0:00:00.199) 0:04:35.697 ******* 2019-12-09T09:39:40.929 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:40.930 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:41.235 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T09:39:41.237 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T09:39:41.282 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:41 +0000 (0:00:00.401) 0:04:36.098 ******* 2019-12-09T09:39:41.289 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:41.290 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:41.537 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:41.823 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:39:41.829 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:41.859 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:41.861 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:42.154 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:42.191 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:42 +0000 (0:00:00.908) 0:04:37.006 ******* 2019-12-09T09:39:42.306 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:42.527 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:39:42.565 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:42 +0000 (0:00:00.374) 0:04:37.381 ******* 2019-12-09T09:39:42.609 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T09:39:42.611 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T09:39:42.674 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:42 +0000 (0:00:00.109) 0:04:37.490 ******* 2019-12-09T09:39:42.791 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:43.285 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:43.418 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T09:39:43.420 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true} 2019-12-09T09:39:43.478 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:43 +0000 (0:00:00.799) 0:04:38.290 ******* 2019-12-09T09:39:43.782 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:39:43.784 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:39:43.832 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:43 +0000 (0:00:00.355) 0:04:38.645 ******* 2019-12-09T09:39:44.152 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T09:39:44.189 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.359) 0:04:39.005 ******* 2019-12-09T09:39:44.191 INFO:teuthology.task.ansible.out: 2019-12-09T09:39:44.207 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T09:39:44.207 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T09:39:44.247 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.057) 0:04:39.062 ******* 2019-12-09T09:39:44.253 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:44.254 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.276 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-09T09:39:44.278 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.315 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.068) 0:04:39.131 ******* 2019-12-09T09:39:44.342 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.380 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.064) 0:04:39.195 ******* 2019-12-09T09:39:44.407 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-09T09:39:44.410 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.455 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.075) 0:04:39.271 ******* 2019-12-09T09:39:44.482 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-09T09:39:44.484 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.523 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.068) 0:04:39.339 ******* 2019-12-09T09:39:44.556 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:44.557 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:44.559 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.597 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.073) 0:04:39.413 ******* 2019-12-09T09:39:44.637 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-09T09:39:44.639 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.674 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.077) 0:04:39.490 ******* 2019-12-09T09:39:44.701 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T09:39:44.703 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:44.739 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:44 +0000 (0:00:00.064) 0:04:39.555 ******* 2019-12-09T09:39:44.865 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:44.867 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:45.200 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:45.335 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} Monday 09 December 2019 09:39:45 +0000 (0:00:00.568) 0:04:40.123 ******* TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:45.374 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:45 +0000 (0:00:00.066) 0:04:40.190 ******* 2019-12-09T09:39:45.413 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T09:39:45.415 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:45.459 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:45 +0000 (0:00:00.084) 0:04:40.274 ******* 2019-12-09T09:39:45.519 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:39:45.521 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:39:45.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:45 +0000 (0:00:00.105) 0:04:40.380 ******* 2019-12-09T09:39:45.888 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-09T09:39:45.889 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:39:45.925 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:45 +0000 (0:00:00.360) 0:04:40.740 ******* 2019-12-09T09:39:45.962 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T09:39:45.963 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T09:39:46.001 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:45 +0000 (0:00:00.075) 0:04:40.816 ******* 2019-12-09T09:39:46.456 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T09:39:46.457 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T09:39:46.503 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:46 +0000 (0:00:00.502) 0:04:41.319 ******* 2019-12-09T09:39:46.509 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T09:39:46.510 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true} 2019-12-09T09:39:46.566 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:46 +0000 (0:00:00.062) 0:04:41.381 ******* 2019-12-09T09:39:46.572 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T09:39:46.573 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:46.618 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:46 +0000 (0:00:00.052) 0:04:41.434 ******* 2019-12-09T09:39:47.034 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T09:39:47.036 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T09:39:47.087 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.468) 0:04:41.903 ******* 2019-12-09T09:39:47.129 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.176 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.081) 0:04:41.984 ******* TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T09:39:47.178 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target sntp.service systemd-journald.socket tmp.mount ntpdate.service system.slice -.mount syslog.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T09:39:47.180 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "-.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T09:39:47.182 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:39:47.225 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.056) 0:04:42.041 ******* 2019-12-09T09:39:47.233 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-09T09:39:47.235 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.277 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.052) 0:04:42.093 ******* 2019-12-09T09:39:47.310 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-09T09:39:47.312 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.368 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.090) 0:04:42.184 ******* 2019-12-09T09:39:47.409 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-09T09:39:47.411 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.477 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.104) 0:04:42.288 ******* 2019-12-09T09:39:47.503 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-09T09:39:47.505 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.541 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.068) 0:04:42.356 ******* 2019-12-09T09:39:47.581 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.616 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.068) 0:04:42.425 ******* 2019-12-09T09:39:47.640 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T09:39:47.642 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:47.677 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:47 +0000 (0:00:00.066) 0:04:42.492 ******* 2019-12-09T09:39:48.132 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T09:39:48.168 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.491) 0:04:42.983 ******* 2019-12-09T09:39:48.182 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-09T09:39:48.184 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T09:39:48.231 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.062) 0:04:43.046 ******* 2019-12-09T09:39:48.238 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-09T09:39:48.239 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:48.278 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.047) 0:04:43.094 ******* 2019-12-09T09:39:48.286 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T09:39:48.287 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:48.326 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.048) 0:04:43.142 ******* 2019-12-09T09:39:48.334 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:48.376 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.046) 0:04:43.188 ******* 2019-12-09T09:39:48.381 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T09:39:48.383 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:48.437 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.064) 0:04:43.253 ******* 2019-12-09T09:39:48.714 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T09:39:48.757 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:48 +0000 (0:00:00.319) 0:04:43.572 ******* 2019-12-09T09:39:49.430 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T09:39:49.431 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T09:39:49.479 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:49 +0000 (0:00:00.722) 0:04:44.295 ******* 2019-12-09T09:39:49.507 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T09:39:49.508 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:49.562 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:49 +0000 (0:00:00.082) 0:04:44.378 ******* 2019-12-09T09:39:49.697 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-09T09:39:49.698 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T09:39:49.768 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:49 +0000 (0:00:00.198) 0:04:44.577 ******* 2019-12-09T09:39:50.104 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T09:39:50.106 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target syslog.target ntpdate.service systemd-journald.socket sntp.service -.mount tmp.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target chronyd.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T09:39:50.107 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T09:39:50.109 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:39:50.143 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:50 +0000 (0:00:00.382) 0:04:44.959 ******* 2019-12-09T09:39:51.070 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T09:39:51.117 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:51 +0000 (0:00:00.974) 0:04:45.933 ******* 2019-12-09T09:39:51.146 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:51.182 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:51 +0000 (0:00:00.064) 0:04:45.998 ******* 2019-12-09T09:39:51.210 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:51.247 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:51 +0000 (0:00:00.064) 0:04:46.062 ******* 2019-12-09T09:39:51.557 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-09T09:39:51.686 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} Monday 09 December 2019 09:39:51 +0000 (0:00:00.357) 0:04:46.420 ******* TASK [testnode : Install packages via pip] ************************************* ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} Monday 09 December 2019 09:39:51 +0000 (0:00:00.067) 0:04:46.488 ******* 2019-12-09T09:39:51.710 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-09T09:39:51.712 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:51.752 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:51 +0000 (0:00:00.079) 0:04:46.568 ******* 2019-12-09T09:39:51.817 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-09T09:39:51.873 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:51 +0000 (0:00:00.121) 0:04:46.689 ******* 2019-12-09T09:39:52.215 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-09T09:39:52.216 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T09:39:52.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:52 +0000 (0:00:00.382) 0:04:47.072 ******* 2019-12-09T09:39:52.295 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T09:39:52.296 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:52.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:52 +0000 (0:00:00.083) 0:04:47.156 ******* 2019-12-09T09:39:52.372 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T09:39:52.373 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T09:39:52.414 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:52 +0000 (0:00:00.073) 0:04:47.229 ******* 2019-12-09T09:39:52.469 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-09T09:39:52.472 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:52.513 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:52 +0000 (0:00:00.098) 0:04:47.328 ******* 2019-12-09T09:39:52.522 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T09:39:52.565 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:52 +0000 (0:00:00.052) 0:04:47.380 ******* 2019-12-09T09:39:52.845 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T09:39:52.892 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:52 +0000 (0:00:00.327) 0:04:47.708 ******* 2019-12-09T09:39:53.186 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T09:39:53.188 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T09:39:53.226 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.334) 0:04:48.042 ******* 2019-12-09T09:39:53.253 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T09:39:53.254 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.293 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.067) 0:04:48.109 ******* 2019-12-09T09:39:53.343 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-09T09:39:53.345 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.346 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.358 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.368 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.369 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.417 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.123) 0:04:48.232 ******* 2019-12-09T09:39:53.455 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi161.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.467 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.468 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.480 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.481 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.518 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.101) 0:04:48.334 ******* 2019-12-09T09:39:53.545 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.585 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.066) 0:04:48.400 ******* 2019-12-09T09:39:53.612 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T09:39:53.614 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.653 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.068) 0:04:48.469 ******* 2019-12-09T09:39:53.681 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T09:39:53.683 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.723 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.069) 0:04:48.538 ******* 2019-12-09T09:39:53.752 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.789 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.066) 0:04:48.604 ******* 2019-12-09T09:39:53.816 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-09T09:39:53.817 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.856 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.067) 0:04:48.672 ******* 2019-12-09T09:39:53.883 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T09:39:53.884 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:53.924 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:53 +0000 (0:00:00.067) 0:04:48.740 ******* 2019-12-09T09:39:53.950 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-09T09:39:53.951 INFO:teuthology.task.ansible.out:skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:54.010 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.086) 0:04:48.826 ******* 2019-12-09T09:39:54.038 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-12-09T09:39:54.038 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T09:39:54.040 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi161.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:54.076 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.066) 0:04:48.892 ******* 2019-12-09T09:39:54.120 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.044) 0:04:48.936 ******* 2019-12-09T09:39:54.126 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-09T09:39:54.126 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:54.165 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.044) 0:04:48.981 ******* 2019-12-09T09:39:54.202 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-09T09:39:54.204 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2019-12-09T09:39:54.242 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.076) 0:04:49.057 ******* 2019-12-09T09:39:54.486 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:39:54.487 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013841", "end": "2019-12-09 09:39:53.870957", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 09:39:53.857116", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:54.574 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-09T09:39:54.576 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T09:39:54.612 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.370) 0:04:49.428 ******* 2019-12-09T09:39:54.644 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T09:39:54.646 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:54.683 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.071) 0:04:49.499 ******* 2019-12-09T09:39:54.710 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T09:39:54.712 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T09:39:54.758 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.074) 0:04:49.574 ******* 2019-12-09T09:39:54.785 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:39:54.786 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006520", "end": "2019-12-09 09:39:54.164649", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 09:39:54.158129", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:54.788 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:54.826 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:54 +0000 (0:00:00.068) 0:04:49.642 ******* 2019-12-09T09:39:55.107 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:39:55.107 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007278", "end": "2019-12-09 09:39:54.494945", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:39:54.487667", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:55.171 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-09T09:39:55.172 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T09:39:55.210 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.384) 0:04:50.026 ******* 2019-12-09T09:39:55.431 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:39:55.432 INFO:teuthology.task.ansible.out:ok: [smithi161.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.006980", "end": "2019-12-09 09:39:54.820796", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 09:39:54.813816", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:55.476 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.265) 0:04:50.292 ******* 2019-12-09T09:39:55.503 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T09:39:55.560 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.083) 0:04:50.375 ******* 2019-12-09T09:39:55.567 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T09:39:55.567 INFO:teuthology.task.ansible.out:changed: [smithi059.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T09:39:55.618 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.058) 0:04:50.434 ******* 2019-12-09T09:39:55.646 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T09:39:55.647 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.688 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.069) 0:04:50.503 ******* 2019-12-09T09:39:55.725 INFO:teuthology.task.ansible.out: 2019-12-09T09:39:55.728 INFO:teuthology.task.ansible.out:TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi059.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.739 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.740 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.751 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.792 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.104) 0:04:50.608 ******* 2019-12-09T09:39:55.834 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi059.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.846 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi059.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.857 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.902 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.109) 0:04:50.718 ******* 2019-12-09T09:39:55.908 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-09T09:39:55.909 INFO:teuthology.task.ansible.out:changed: [smithi161.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:39:55.948 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:39:55.950 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:55.991 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:55 +0000 (0:00:00.088) 0:04:50.806 ******* 2019-12-09T09:39:56.031 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T09:39:56.033 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:56.080 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:56 +0000 (0:00:00.088) 0:04:50.895 ******* 2019-12-09T09:39:56.110 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T09:39:56.111 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:56.154 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:56 +0000 (0:00:00.074) 0:04:50.970 ******* 2019-12-09T09:39:56.181 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T09:39:56.183 INFO:teuthology.task.ansible.out:skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:39:56.239 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:56 +0000 (0:00:00.085) 0:04:51.055 ******* 2019-12-09T09:39:56.590 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 09 December 2019 09:39:56 +0000 (0:00:00.079) 0:04:51.135 ******* TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 09 December 2019 09:39:56 +0000 (0:00:00.064) 0:04:51.200 ******* TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 09 December 2019 09:39:56 +0000 (0:00:00.082) 0:04:51.282 ******* TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi059.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 09 December 2019 09:39:56 +0000 (0:00:00.078) 0:04:51.360 ******* 2019-12-09T09:39:56.899 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi059.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016372", "end": "2019-12-09 09:39:56.844854", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 09:39:56.828482", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:57.214 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007738", "end": "2019-12-09 09:39:57.173459", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 09:39:57.165721", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:57.524 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007363", "end": "2019-12-09 09:39:57.482197", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:39:57.474834", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:57.838 INFO:teuthology.task.ansible.out:ok: [smithi059.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008781", "end": "2019-12-09 09:39:57.804002", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 09:39:57.795221", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:39:57.919 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:57 +0000 (0:00:01.348) 0:04:52.709 ******* 2019-12-09T09:39:57.929 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T09:39:57.976 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:57 +0000 (0:00:00.082) 0:04:52.792 ******* 2019-12-09T09:39:58.310 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi059.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:39:58.416 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched PLAY [paddles] ***************************************************************** skipping: no hosts matched 2019-12-09T09:39:58.418 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi059.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 smithi161.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 2019-12-09T09:39:58.419 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:39:58 +0000 (0:00:00.441) 0:04:53.234 ******* =============================================================================== users : Create all admin users with sudo access. ----------------------- 35.84s users : Update authorized_keys using the keys repo --------------------- 34.20s common : nrpe - Ensure SELinux policy is up to date -------------------- 25.77s testnode : Install packages -------------------------------------------- 15.10s users : Remove revoked users ------------------------------------------- 14.89s common : Update yum cache ---------------------------------------------- 10.63s common : nrpe - Remove SELinux policy package --------------------------- 6.18s common : nrpe - Load SELinux policy package ----------------------------- 5.72s testnode : Ensure ceph packages are not present. ------------------------ 5.23s testnode : Zap all non-root disks --------------------------------------- 5.13s common : Make sure smartmontools is installed --------------------------- 3.61s ansible-managed : Add authorized keys for the ansible user. ------------- 3.00s Gathering Facts --------------------------------------------------------- 2.92s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.68s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 2.26s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 2.03s testnode : Rebuild rpmdb ------------------------------------------------ 1.69s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.55s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.35s common : nrpe - Copy SELinux type enforcement file ---------------------- 1.02s 2019-12-09T09:39:58.713 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2019-12-09T09:39:58.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:39:58.716 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi059.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:39:58.816 INFO:teuthology.orchestra.run.smithi059:> true 2019-12-09T09:39:59.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:39:59.222 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi161.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:39:59.321 INFO:teuthology.orchestra.run.smithi161:> true 2019-12-09T09:39:59.708 INFO:teuthology.run_tasks:Running task clock... 2019-12-09T09:39:59.721 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-12-09T09:39:59.721 INFO:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T09:39:59.862 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T09:40:09.955 INFO:teuthology.orchestra.run.smithi059.stdout:ntpd: time slew -0.000404s 2019-12-09T09:40:10.001 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T09:40:10.097 INFO:teuthology.orchestra.run.smithi059.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T09:40:10.097 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================== 2019-12-09T09:40:10.097 INFO:teuthology.orchestra.run.smithi059.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:10.097 INFO:teuthology.orchestra.run.smithi059.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:10.097 INFO:teuthology.orchestra.run.smithi059.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:10.098 INFO:teuthology.orchestra.run.smithi059.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:10.098 INFO:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T09:40:10.249 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T09:40:20.332 INFO:teuthology.orchestra.run.smithi161.stdout:ntpd: time set +0.574870s 2019-12-09T09:40:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T09:40:20.445 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T09:40:20.445 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2019-12-09T09:40:20.445 INFO:teuthology.orchestra.run.smithi161.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:20.445 INFO:teuthology.orchestra.run.smithi161.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:20.445 INFO:teuthology.orchestra.run.smithi161.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:20.445 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:40:20.447 INFO:teuthology.run_tasks:Running task install... 2019-12-09T09:40:20.508 DEBUG:teuthology.task.install:project ceph 2019-12-09T09:40:20.508 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}} 2019-12-09T09:40:20.508 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'} 2019-12-09T09:40:20.508 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-12-09T09:40:20.513 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-12-09T09:40:20.513 INFO:teuthology.task.install:extra packages: [] 2019-12-09T09:40:20.513 INFO:teuthology.orchestra.run.smithi059:> sudo lsb_release -is 2019-12-09T09:40:20.692 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS 2019-12-09T09:40:20.704 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:40:20.705 INFO:teuthology.orchestra.run.smithi161:> sudo lsb_release -is 2019-12-09T09:40:20.705 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T09:40:20.706 INFO:teuthology.orchestra.run.smithi059:> sudo lsb_release -is 2019-12-09T09:40:20.862 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS 2019-12-09T09:40:20.864 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:40:20.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:40:20.875 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS 2019-12-09T09:40:20.877 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:40:20.878 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T09:40:20.878 INFO:teuthology.orchestra.run.smithi161:> sudo lsb_release -is 2019-12-09T09:40:20.998 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS 2019-12-09T09:40:21.000 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:40:21.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:40:21.020 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T09:40:21.020 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T09:40:21.154 INFO:teuthology.orchestra.run.smithi059:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T09:40:21.169 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T09:40:21.169 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T09:40:21.284 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T09:40:21.284 INFO:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T09:40:21.289 INFO:teuthology.orchestra.run.smithi161:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T09:40:21.391 INFO:teuthology.orchestra.run.smithi059:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T09:40:21.408 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T09:40:21.408 INFO:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T09:40:21.504 INFO:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T09:40:21.519 INFO:teuthology.orchestra.run.smithi161:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T09:40:21.597 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2019-12-09T09:40:21.599 INFO:teuthology.orchestra.run.smithi059:> sudo yum clean all 2019-12-09T09:40:21.624 INFO:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T09:40:21.724 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2019-12-09T09:40:21.727 INFO:teuthology.orchestra.run.smithi161:> sudo yum clean all 2019-12-09T09:40:21.780 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:40:21.825 INFO:teuthology.orchestra.run.smithi059.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T09:40:21.825 INFO:teuthology.orchestra.run.smithi059.stdout: : lab-extras updates 2019-12-09T09:40:21.867 INFO:teuthology.orchestra.run.smithi059.stdout:Cleaning up list of fastest mirrors 2019-12-09T09:40:21.868 INFO:teuthology.orchestra.run.smithi059.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T09:40:21.880 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw 2019-12-09T09:40:21.946 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:40:21.989 INFO:teuthology.orchestra.run.smithi161.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T09:40:21.989 INFO:teuthology.orchestra.run.smithi161.stdout: : lab-extras updates 2019-12-09T09:40:22.036 INFO:teuthology.orchestra.run.smithi161.stdout:Cleaning up list of fastest mirrors 2019-12-09T09:40:22.036 INFO:teuthology.orchestra.run.smithi161.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T09:40:22.049 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw 2019-12-09T09:40:22.070 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:40:22.109 INFO:teuthology.orchestra.run.smithi059.stdout:Determining fastest mirrors 2019-12-09T09:40:22.286 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:40:22.329 INFO:teuthology.orchestra.run.smithi161.stdout:Determining fastest mirrors 2019-12-09T09:40:22.573 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:40:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:40:22.574 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:40:22.575 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:40:22.873 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:40:22.874 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:40:22.874 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:40:22.875 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:40:29.718 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:40:30.085 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:40:30.301 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:40:30.301 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.306 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.306 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.306 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.306 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:40:30.307 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.308 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.308 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.308 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.308 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.311 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.311 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.311 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T09:40:30.311 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T09:40:30.311 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:40:30.311 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.312 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.313 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T09:40:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.315 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.316 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T09:40:30.316 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.316 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.316 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:30.316 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T09:40:30.728 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:40:31.083 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:40:31.083 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:40:31.083 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.096 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.165 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:40:31.203 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.204 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.205 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.207 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.207 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.209 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.210 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.211 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.212 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:40:31.212 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.220 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.229 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.229 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.229 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.230 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.230 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.230 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T09:40:31.231 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T09:40:31.231 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:40:31.231 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:40:31.232 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T09:40:31.233 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.234 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:40:31.234 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.238 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.239 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.240 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.241 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.242 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.243 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.244 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.245 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.246 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.246 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.247 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:40:31.248 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T09:40:31.248 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.248 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:40:31.249 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:40:31.249 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T09:40:31.250 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:40:31.251 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:40:31.251 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:40:31.251 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T09:40:31.253 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.253 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.254 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.254 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.255 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T09:40:31.255 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.256 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:40:31.256 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:40:31.256 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.256 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:40:31.257 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T09:40:31.259 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:40:31.259 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:40:31.260 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:40:31.260 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:40:31.260 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:40:31.260 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:40:31.260 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T09:40:31.260 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T09:40:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T09:40:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T09:40:31.263 INFO:teuthology.orchestra.run.smithi059.stdout:Updating for dependencies: 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T09:40:31.264 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T09:40:31.265 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:40:31.265 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 45 M 2019-12-09T09:40:31.265 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:40:31.265 INFO:teuthology.orchestra.run.smithi059.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:40:32.121 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:40:32.208 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:40:32.208 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:40:32.213 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T09:40:32.214 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T09:40:32.215 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T09:40:32.216 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T09:40:32.216 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T09:40:32.216 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T09:40:32.216 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T09:40:32.218 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T09:40:32.218 INFO:teuthology.orchestra.run.smithi161.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T09:40:32.218 INFO:teuthology.orchestra.run.smithi161.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T09:40:32.218 INFO:teuthology.orchestra.run.smithi161.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T09:40:32.218 INFO:teuthology.orchestra.run.smithi161.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T09:40:32.218 INFO:teuthology.orchestra.run.smithi161.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T09:40:32.223 INFO:teuthology.orchestra.run.smithi161.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T09:40:32.223 INFO:teuthology.orchestra.run.smithi161.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T09:40:32.224 INFO:teuthology.orchestra.run.smithi161.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T09:40:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Updating for dependencies: 2019-12-09T09:40:32.224 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T09:40:32.224 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T09:40:32.224 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:40:32.224 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 45 M 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:40:32.225 INFO:teuthology.orchestra.run.smithi161.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:40:33.844 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:40:33.844 INFO:teuthology.orchestra.run.smithi059.stdout:Total 17 MB/s | 45 MB 00:02 2019-12-09T09:40:33.860 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:40:33.930 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:40:34.163 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:40:34.170 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:40:35.062 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T09:40:35.208 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:40:35.208 INFO:teuthology.orchestra.run.smithi161.stdout:Total 15 MB/s | 45 MB 00:02 2019-12-09T09:40:35.226 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:40:35.292 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:40:35.391 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T09:40:35.536 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:40:35.537 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:40:35.748 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T09:40:36.083 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:40:36.256 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T09:40:36.384 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:40:36.624 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T09:40:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T09:40:36.998 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T09:40:37.344 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:40:37.566 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T09:40:37.609 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:40:37.939 INFO:teuthology.orchestra.run.smithi059.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T09:40:37.994 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T09:40:38.803 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T09:40:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:40:39.171 INFO:teuthology.orchestra.run.smithi161.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T09:40:39.272 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:40:39.817 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T09:40:40.044 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:40:40.300 INFO:teuthology.orchestra.run.smithi059.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:40:40.438 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:40:40.555 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T09:40:40.787 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T09:40:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T09:40:41.111 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:40:41.327 INFO:teuthology.orchestra.run.smithi161.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:40:41.378 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:40:41.622 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T09:40:41.643 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T09:40:41.888 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T09:40:42.002 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T09:40:42.202 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:40:42.496 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:40:42.810 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T09:40:43.184 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T09:40:44.464 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T09:40:45.596 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:40:45.641 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T09:40:46.151 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T09:40:46.806 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:40:47.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T09:40:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:40:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:40:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:40:56.926 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:40:57.169 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:40:57.288 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T09:40:57.303 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:40:57.393 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T09:40:57.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T09:40:57.586 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:40:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:40:57.723 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:40:57.857 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T09:40:57.908 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:40:57.966 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T09:40:58.040 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T09:40:58.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T09:40:58.167 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:40:58.169 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T09:40:58.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:40:58.307 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T09:40:58.343 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T09:40:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:40:58.444 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:40:58.525 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:40:58.553 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T09:40:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T09:40:58.661 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T09:40:58.726 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T09:40:58.762 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:40:58.857 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:40:58.862 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T09:40:58.944 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:40:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T09:40:59.044 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:40:59.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T09:40:59.187 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T09:40:59.189 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T09:40:59.296 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:40:59.298 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:40:59.404 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T09:40:59.530 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T09:40:59.630 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:40:59.739 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:40:59.890 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T09:41:00.015 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T09:41:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T09:41:00.116 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T09:41:00.216 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:41:00.249 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:41:00.325 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:41:00.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:41:00.358 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T09:41:00.451 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:41:00.576 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:41:00.578 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:41:00.689 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:41:00.689 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:00.689 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:00.689 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.689 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:00.689 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.690 INFO:teuthology.orchestra.run.smithi059.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.691 INFO:teuthology.orchestra.run.smithi059.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T09:41:00.692 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:00.693 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Updated: 2019-12-09T09:41:00.693 INFO:teuthology.orchestra.run.smithi059.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.693 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:00.693 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:00.693 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:00.817 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-test 2019-12-09T09:41:01.136 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:01.183 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:01.184 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:01.184 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:01.184 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:01.185 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:01.357 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:41:01.357 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:01.357 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:01.357 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T09:41:01.358 INFO:teuthology.orchestra.run.smithi161.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.359 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T09:41:01.360 INFO:teuthology.orchestra.run.smithi161.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.360 INFO:teuthology.orchestra.run.smithi161.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.360 INFO:teuthology.orchestra.run.smithi161.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.360 INFO:teuthology.orchestra.run.smithi161.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T09:41:01.360 INFO:teuthology.orchestra.run.smithi161.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.360 INFO:teuthology.orchestra.run.smithi161.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.361 INFO:teuthology.orchestra.run.smithi161.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.361 INFO:teuthology.orchestra.run.smithi161.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T09:41:01.361 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:01.361 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Updated: 2019-12-09T09:41:01.361 INFO:teuthology.orchestra.run.smithi161.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.362 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:01.362 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:01.362 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:01.472 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-test 2019-12-09T09:41:01.734 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:01.773 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:01.857 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:02.208 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:02.209 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:02.209 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:02.228 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:02.259 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:02.259 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:02.261 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:02.261 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T09:41:02.262 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T09:41:02.262 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T09:41:02.264 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T09:41:02.267 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:02.267 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T09:41:02.418 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:02.649 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:02.748 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:02.749 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:02.749 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:02.767 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:02.802 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:02.802 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:02.802 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:02.802 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T09:41:02.803 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T09:41:02.803 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T09:41:02.803 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T09:41:02.804 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:02.805 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T09:41:02.878 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:02.878 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:02.879 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:02.880 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:02.880 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:41:02.880 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:41:02.881 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:02.881 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:02.881 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T09:41:02.882 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:41:02.882 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T09:41:02.882 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T09:41:02.882 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T09:41:02.882 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T09:41:02.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:02.883 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:02.883 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:02.883 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:41:02.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:02.884 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 38 M 2019-12-09T09:41:02.884 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 134 M 2019-12-09T09:41:02.884 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:03.164 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:03.386 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:03.387 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:03.387 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T09:41:03.388 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:03.389 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:03.390 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:41:03.390 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:03.390 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 38 M 2019-12-09T09:41:03.391 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 134 M 2019-12-09T09:41:03.391 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:04.738 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:04.738 INFO:teuthology.orchestra.run.smithi059.stdout:Total 21 MB/s | 38 MB 00:01 2019-12-09T09:41:04.742 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:04.770 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:04.864 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:04.864 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:05.494 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T09:41:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T09:41:06.099 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T09:41:06.457 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:06.457 INFO:teuthology.orchestra.run.smithi161.stdout:Total 12 MB/s | 38 MB 00:03 2019-12-09T09:41:06.460 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:41:06.480 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:41:06.533 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:41:06.534 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:41:07.278 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T09:41:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T09:41:07.909 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T09:41:10.305 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:41:10.720 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:41:10.808 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:41:10.925 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T09:41:11.029 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T09:41:11.117 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:41:11.544 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T09:41:11.545 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:11.545 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:11.545 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:11.545 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:11.545 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:41:11.546 INFO:teuthology.orchestra.run.smithi059.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T09:41:11.546 INFO:teuthology.orchestra.run.smithi059.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T09:41:11.546 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:11.546 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:11.679 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph 2019-12-09T09:41:11.927 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:41:11.932 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:11.977 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:11.978 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:11.978 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:11.978 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:11.979 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:12.261 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:41:12.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:41:12.468 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T09:41:12.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T09:41:12.664 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:41:13.011 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:13.012 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.012 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:13.014 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.045 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.046 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.047 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.049 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.049 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:13.059 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:13.061 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.062 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.065 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.065 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.066 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.069 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:13.069 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:13.071 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:13.076 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.076 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T09:41:13.077 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T09:41:13.078 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T09:41:13.079 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:13.079 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:13.080 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:13.081 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:13.082 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:13.083 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T09:41:13.084 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T09:41:13.085 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T09:41:13.085 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T09:41:13.085 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T09:41:13.086 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.086 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T09:41:13.087 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:41:13.087 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:41:13.088 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T09:41:13.088 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T09:41:13.089 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T09:41:13.089 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T09:41:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T09:41:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T09:41:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T09:41:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T09:41:13.091 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T09:41:13.091 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.092 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T09:41:13.092 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T09:41:13.092 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:13.092 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T09:41:13.262 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T09:41:13.262 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:13.262 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:13.262 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:13.263 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:13.263 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:41:13.263 INFO:teuthology.orchestra.run.smithi161.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T09:41:13.263 INFO:teuthology.orchestra.run.smithi161.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T09:41:13.263 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:13.263 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:13.381 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph 2019-12-09T09:41:13.473 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:13.621 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:13.661 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:13.662 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:13.663 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:13.664 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:13.664 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:13.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:13.704 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:13.707 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:41:13.707 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:41:13.707 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:13.707 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T09:41:13.708 INFO:teuthology.orchestra.run.smithi059.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T09:41:13.709 INFO:teuthology.orchestra.run.smithi059.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T09:41:13.710 INFO:teuthology.orchestra.run.smithi059.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T09:41:13.711 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:13.712 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 28 M 2019-12-09T09:41:13.712 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 87 M 2019-12-09T09:41:13.712 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:14.331 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:14.666 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:14.667 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.667 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:14.669 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.699 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.700 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.701 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.703 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.703 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:14.712 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:14.714 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.715 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.717 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.718 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.719 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.720 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:41:14.721 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:14.724 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:14.728 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.728 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T09:41:14.729 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T09:41:14.730 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T09:41:14.730 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:14.731 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:14.732 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:14.733 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:14.734 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:41:14.735 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T09:41:14.735 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T09:41:14.736 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T09:41:14.736 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T09:41:14.737 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T09:41:14.738 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.738 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T09:41:14.738 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:41:14.739 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:41:14.740 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T09:41:14.740 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T09:41:14.740 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T09:41:14.742 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T09:41:14.742 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T09:41:14.742 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T09:41:14.743 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.743 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T09:41:14.743 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T09:41:14.743 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T09:41:14.743 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.744 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T09:41:14.744 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T09:41:14.744 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:14.745 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T09:41:15.047 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:15.048 INFO:teuthology.orchestra.run.smithi059.stdout:Total 21 MB/s | 28 MB 00:01 2019-12-09T09:41:15.064 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:15.105 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:15.119 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:15.323 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:15.323 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:15.326 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:15.326 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:41:15.327 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T09:41:15.328 INFO:teuthology.orchestra.run.smithi161.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T09:41:15.329 INFO:teuthology.orchestra.run.smithi161.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T09:41:15.330 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:15.331 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:15.331 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:15.331 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T09:41:15.331 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:15.332 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 28 M 2019-12-09T09:41:15.332 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 87 M 2019-12-09T09:41:15.333 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:15.333 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:15.334 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:16.178 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T09:41:16.498 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T09:41:16.871 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T09:41:17.285 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:17.286 INFO:teuthology.orchestra.run.smithi161.stdout:Total 14 MB/s | 28 MB 00:01 2019-12-09T09:41:17.310 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:41:17.357 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:41:17.369 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T09:41:17.618 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:41:17.619 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:41:17.725 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T09:41:18.156 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T09:41:18.395 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T09:41:18.645 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T09:41:18.764 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T09:41:19.064 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T09:41:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T09:41:19.925 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T09:41:20.397 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T09:41:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T09:41:20.915 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T09:41:20.980 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T09:41:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T09:41:21.696 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T09:41:22.465 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T09:41:22.967 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T09:41:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:41:23.370 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T09:41:23.374 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T09:41:23.786 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T09:41:23.942 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T09:41:24.130 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T09:41:24.396 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T09:41:24.651 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T09:41:24.900 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T09:41:24.931 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T09:41:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T09:41:25.357 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:41:25.691 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T09:41:26.057 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T09:41:26.279 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T09:41:26.463 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T09:41:26.622 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T09:41:26.720 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T09:41:26.845 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T09:41:26.908 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T09:41:26.995 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T09:41:27.104 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T09:41:27.199 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T09:41:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T09:41:27.296 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T09:41:27.397 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T09:41:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T09:41:27.584 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T09:41:27.588 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T09:41:27.690 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T09:41:27.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T09:41:27.899 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T09:41:28.008 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T09:41:28.109 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:41:28.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T09:41:28.268 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T09:41:28.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T09:41:28.427 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T09:41:28.578 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T09:41:28.694 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T09:41:28.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T09:41:28.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T09:41:28.920 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T09:41:29.017 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T09:41:29.020 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T09:41:29.161 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T09:41:29.245 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T09:41:29.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T09:41:29.457 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T09:41:29.457 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:29.458 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:29.458 INFO:teuthology.orchestra.run.smithi059.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:29.458 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:29.458 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:41:29.459 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:29.459 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:29.459 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:29.459 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:29.460 INFO:teuthology.orchestra.run.smithi059.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T09:41:29.460 INFO:teuthology.orchestra.run.smithi059.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T09:41:29.461 INFO:teuthology.orchestra.run.smithi059.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T09:41:29.461 INFO:teuthology.orchestra.run.smithi059.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T09:41:29.462 INFO:teuthology.orchestra.run.smithi059.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T09:41:29.462 INFO:teuthology.orchestra.run.smithi059.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T09:41:29.462 INFO:teuthology.orchestra.run.smithi059.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T09:41:29.462 INFO:teuthology.orchestra.run.smithi059.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T09:41:29.463 INFO:teuthology.orchestra.run.smithi059.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T09:41:29.463 INFO:teuthology.orchestra.run.smithi059.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T09:41:29.463 INFO:teuthology.orchestra.run.smithi059.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T09:41:29.463 INFO:teuthology.orchestra.run.smithi059.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T09:41:29.464 INFO:teuthology.orchestra.run.smithi059.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T09:41:29.464 INFO:teuthology.orchestra.run.smithi059.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T09:41:29.464 INFO:teuthology.orchestra.run.smithi059.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T09:41:29.464 INFO:teuthology.orchestra.run.smithi059.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T09:41:29.465 INFO:teuthology.orchestra.run.smithi059.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T09:41:29.465 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:29.465 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:29.488 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T09:41:29.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T09:41:29.585 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-daemon 2019-12-09T09:41:29.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T09:41:29.756 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T09:41:29.882 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:29.889 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T09:41:29.929 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:29.930 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:29.931 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:29.931 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:29.931 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:30.009 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T09:41:30.137 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T09:41:30.258 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T09:41:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T09:41:30.502 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:41:30.607 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:30.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T09:41:30.720 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T09:41:30.962 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:30.962 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:30.963 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:31.045 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T09:41:31.187 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T09:41:31.288 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T09:41:31.349 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:31.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T09:41:31.522 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T09:41:31.580 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:31.580 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:31.581 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:31.581 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:31.582 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:31.583 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:31.583 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2019-12-09T09:41:31.583 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:31.583 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 17 k 2019-12-09T09:41:31.583 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 68 k 2019-12-09T09:41:31.583 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:31.615 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T09:41:31.882 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:31.890 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:32.075 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T09:41:32.076 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:32.076 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:32.076 INFO:teuthology.orchestra.run.smithi161.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:32.076 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:32.076 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:41:32.077 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:32.077 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:32.077 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:32.077 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:32.077 INFO:teuthology.orchestra.run.smithi161.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T09:41:32.078 INFO:teuthology.orchestra.run.smithi161.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T09:41:32.078 INFO:teuthology.orchestra.run.smithi161.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T09:41:32.078 INFO:teuthology.orchestra.run.smithi161.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T09:41:32.078 INFO:teuthology.orchestra.run.smithi161.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T09:41:32.078 INFO:teuthology.orchestra.run.smithi161.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T09:41:32.079 INFO:teuthology.orchestra.run.smithi161.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T09:41:32.079 INFO:teuthology.orchestra.run.smithi161.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T09:41:32.079 INFO:teuthology.orchestra.run.smithi161.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T09:41:32.079 INFO:teuthology.orchestra.run.smithi161.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T09:41:32.079 INFO:teuthology.orchestra.run.smithi161.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T09:41:32.080 INFO:teuthology.orchestra.run.smithi161.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T09:41:32.080 INFO:teuthology.orchestra.run.smithi161.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T09:41:32.080 INFO:teuthology.orchestra.run.smithi161.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T09:41:32.080 INFO:teuthology.orchestra.run.smithi161.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T09:41:32.080 INFO:teuthology.orchestra.run.smithi161.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T09:41:32.080 INFO:teuthology.orchestra.run.smithi161.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T09:41:32.081 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:32.081 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:32.195 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-daemon 2019-12-09T09:41:32.488 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:32.531 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:32.531 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:32.532 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:32.532 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:32.533 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:32.681 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:41:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:41:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:33.186 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:33.186 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:33.187 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:33.187 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:33.187 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:33.310 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr 2019-12-09T09:41:33.523 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:33.524 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:33.524 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:33.592 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:33.640 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:33.641 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:33.641 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:33.888 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:34.115 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:34.115 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:34.116 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:41:34.116 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:41:34.116 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2019-12-09T09:41:34.117 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:34.118 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 17 k 2019-12-09T09:41:34.118 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 68 k 2019-12-09T09:41:34.118 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:34.410 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:41:34.411 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:41:34.412 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:41:34.413 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:41:34.683 INFO:teuthology.orchestra.run.smithi059.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:41:34.683 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do 2019-12-09T09:41:34.745 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T09:41:34.913 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:34.948 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:34.949 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:34.949 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:34.950 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:34.950 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:35.233 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:41:35.580 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:35.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:41:35.664 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:35.664 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:35.664 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:35.664 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:35.664 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:35.788 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr 2019-12-09T09:41:35.920 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:35.921 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:35.921 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:35.931 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:35.954 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:35.956 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:35.956 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:35.956 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T09:41:35.956 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T09:41:35.957 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:35.957 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T09:41:36.018 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:36.061 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:36.062 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:36.063 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:36.063 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:36.064 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:36.342 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:36.573 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:36.573 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:36.574 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:36.574 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:36.574 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:41:36.574 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:36.575 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:36.576 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:36.576 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T09:41:36.576 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:36.576 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 4.6 M 2019-12-09T09:41:36.577 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 20 M 2019-12-09T09:41:36.577 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:36.737 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:37.075 INFO:teuthology.orchestra.run.smithi161.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:41:37.075 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do 2019-12-09T09:41:37.137 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T09:41:37.349 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:37.349 INFO:teuthology.orchestra.run.smithi059.stdout:Total 5.9 MB/s | 4.6 MB 00:00 2019-12-09T09:41:37.355 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:37.364 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:37.376 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:37.415 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:37.416 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:37.416 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:37.416 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:37.417 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:37.536 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:37.537 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:38.011 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:38.160 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T09:41:38.340 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:38.340 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:38.341 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:38.344 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:38.374 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:38.375 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:38.375 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:38.375 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T09:41:38.376 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T09:41:38.376 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:38.377 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T09:41:38.565 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T09:41:38.742 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:38.969 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:38.969 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:38.970 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:41:38.970 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T09:41:38.971 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 4.6 M 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 20 M 2019-12-09T09:41:38.972 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:Total 7.3 MB/s | 4.6 MB 00:00 2019-12-09T09:41:39.601 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:41:39.610 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:41:39.727 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T09:41:39.801 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:41:39.801 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:41:40.491 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T09:41:40.933 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T09:41:41.315 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T09:41:41.432 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T09:41:41.558 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T09:41:41.658 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T09:41:42.118 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:42.119 INFO:teuthology.orchestra.run.smithi059.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T09:41:42.120 INFO:teuthology.orchestra.run.smithi059.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T09:41:42.120 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:42.120 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:42.252 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T09:41:42.489 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:42.531 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:42.532 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:42.533 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:42.533 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:42.533 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:42.941 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T09:41:43.201 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T09:41:43.533 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T09:41:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:43.545 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:43.684 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T09:41:43.801 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T09:41:43.930 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:44.161 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:44.161 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:44.162 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:41:44.162 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:44.162 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:44.162 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T09:41:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T09:41:44.163 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:44.163 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:44.163 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:44.164 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2019-12-09T09:41:44.164 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:44.164 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 82 k 2019-12-09T09:41:44.164 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 411 k 2019-12-09T09:41:44.164 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:44.269 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T09:41:44.269 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:44.269 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:44.269 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:44.269 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:44.269 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:41:44.270 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:44.270 INFO:teuthology.orchestra.run.smithi161.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T09:41:44.270 INFO:teuthology.orchestra.run.smithi161.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T09:41:44.270 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:44.270 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:44.389 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T09:41:44.595 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:44.633 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:44.634 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:44.634 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:44.634 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:44.635 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:44.679 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:44.683 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:44.738 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:44.739 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:45.253 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:45.449 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:41:45.583 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:45.583 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:45.583 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:45.888 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:41:45.889 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:45.889 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:45.889 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:45.889 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:45.889 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:45.949 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:46.016 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T09:41:46.173 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:46.174 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:46.174 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T09:41:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 82 k 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 411 k 2019-12-09T09:41:46.176 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:46.249 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:46.295 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:46.296 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:46.297 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:46.297 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:46.297 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:46.503 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:41:46.507 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:41:46.563 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:41:46.563 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:41:46.982 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:47.300 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:41:47.332 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:47.332 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:47.332 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:47.335 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:47.366 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:47.366 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T09:41:47.373 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:41:47.375 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:41:47.377 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:41:47.377 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:47.377 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T09:41:47.378 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T09:41:47.379 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:41:47.381 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:41:47.381 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:47.381 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T09:41:47.764 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:47.848 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:41:47.848 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:47.848 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:47.848 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:47.848 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:47.849 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:47.982 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T09:41:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:47.995 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:47.997 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:41:47.997 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:47.997 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local 2019-12-09T09:41:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:47.998 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:47.999 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:47.999 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:41:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:47.999 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 19 M 2019-12-09T09:41:47.999 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 104 M 2019-12-09T09:41:47.999 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:48.231 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:48.270 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:48.712 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:48.713 INFO:teuthology.orchestra.run.smithi059.stdout:Total 27 MB/s | 19 MB 00:00 2019-12-09T09:41:48.723 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:48.750 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:48.851 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:48.851 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:48.921 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:49.256 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:49.258 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:49.288 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:49.288 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T09:41:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:41:49.296 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:41:49.298 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:41:49.298 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:49.298 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T09:41:49.299 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T09:41:49.300 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:41:49.303 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:41:49.303 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:49.303 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T09:41:49.662 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:49.741 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T09:41:49.885 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:49.886 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:49.887 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:49.887 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:49.887 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:41:49.887 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:49.887 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T09:41:49.888 INFO:teuthology.orchestra.run.smithi161.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 19 M 2019-12-09T09:41:49.889 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 104 M 2019-12-09T09:41:49.890 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:50.198 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T09:41:50.580 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:50.580 INFO:teuthology.orchestra.run.smithi161.stdout:Total 28 MB/s | 19 MB 00:00 2019-12-09T09:41:50.592 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:41:50.622 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:41:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:41:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:41:51.780 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T09:41:52.297 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T09:41:52.408 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T09:41:54.064 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T09:41:54.127 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T09:41:54.448 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T09:41:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T09:41:54.635 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T09:41:54.744 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T09:41:54.837 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T09:41:55.280 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T09:41:55.280 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:55.280 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:41:55.280 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:55.280 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:55.281 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:41:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T09:41:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T09:41:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:55.281 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:41:55.404 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-rook 2019-12-09T09:41:55.672 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:55.719 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T09:41:56.084 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T09:41:56.211 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T09:41:56.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T09:41:56.401 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:56.768 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:41:56.768 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:56.768 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:56.768 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:56.786 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:56.786 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T09:41:56.788 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:56.789 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:56.790 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:56.791 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:56.792 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:56.793 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T09:41:56.793 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T09:41:56.793 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T09:41:56.794 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:41:56.795 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:41:56.796 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:41:56.797 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T09:41:56.797 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T09:41:56.798 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:56.798 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T09:41:56.798 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T09:41:56.799 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T09:41:56.807 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T09:41:56.807 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T09:41:56.807 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:41:56.807 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T09:41:57.010 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T09:41:57.105 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T09:41:57.184 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:57.418 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:57.418 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:41:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:57.420 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:57.420 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:41:57.420 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:57.420 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:41:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T09:41:57.421 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:41:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T09:41:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T09:41:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T09:41:57.421 INFO:teuthology.orchestra.run.smithi059.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T09:41:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T09:41:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T09:41:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T09:41:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T09:41:57.422 INFO:teuthology.orchestra.run.smithi059.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T09:41:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T09:41:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:57.423 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:41:57.423 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:41:57.423 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T09:41:57.423 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:41:57.424 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 1.6 M 2019-12-09T09:41:57.424 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 24 M 2019-12-09T09:41:57.424 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:41:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T09:41:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:57.641 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:41:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:41:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:57.641 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:41:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T09:41:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T09:41:57.642 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:57.642 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:41:57.768 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-rook 2019-12-09T09:41:58.021 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:41:58.061 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:41:58.061 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:41:58.062 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:41:58.062 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:41:58.063 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:41:58.226 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:41:58.226 INFO:teuthology.orchestra.run.smithi059.stdout:Total 2.0 MB/s | 1.6 MB 00:00 2019-12-09T09:41:58.241 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:41:58.253 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:41:58.322 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:41:58.323 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:41:58.736 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:41:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:41:59.066 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:59.066 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:41:59.067 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:41:59.071 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T09:41:59.098 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:59.098 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T09:41:59.100 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:59.101 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:59.102 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:59.103 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:41:59.104 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:59.104 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T09:41:59.105 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T09:41:59.105 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T09:41:59.105 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:41:59.106 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:41:59.107 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:41:59.108 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T09:41:59.108 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T09:41:59.109 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:59.110 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T09:41:59.110 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T09:41:59.117 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T09:41:59.118 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T09:41:59.118 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T09:41:59.118 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:41:59.118 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T09:41:59.440 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T09:41:59.481 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:41:59.709 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:59.709 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:41:59.710 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:41:59.711 INFO:teuthology.orchestra.run.smithi161.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T09:41:59.712 INFO:teuthology.orchestra.run.smithi161.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T09:41:59.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:41:59.714 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 1.6 M 2019-12-09T09:41:59.714 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 24 M 2019-12-09T09:41:59.714 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:41:59.783 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T09:42:00.037 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T09:42:00.363 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T09:42:00.501 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:00.501 INFO:teuthology.orchestra.run.smithi161.stdout:Total 2.1 MB/s | 1.6 MB 00:00 2019-12-09T09:42:00.515 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:42:00.528 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:42:00.597 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:42:00.597 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:42:00.867 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T09:42:01.257 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T09:42:01.332 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T09:42:01.591 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T09:42:01.938 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T09:42:02.341 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T09:42:02.654 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T09:42:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T09:42:03.029 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T09:42:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T09:42:03.407 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T09:42:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T09:42:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T09:42:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T09:42:03.801 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T09:42:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T09:42:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T09:42:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T09:42:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T09:42:04.140 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T09:42:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T09:42:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T09:42:04.444 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T09:42:04.536 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T09:42:05.038 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T09:42:05.038 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T09:42:05.039 INFO:teuthology.orchestra.run.smithi059.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T09:42:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T09:42:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T09:42:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T09:42:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T09:42:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T09:42:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T09:42:05.041 INFO:teuthology.orchestra.run.smithi059.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T09:42:05.041 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:05.041 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:42:05.159 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-ssh 2019-12-09T09:42:05.369 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:05.409 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:05.409 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:05.410 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:05.410 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:05.411 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:05.480 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T09:42:05.863 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T09:42:06.063 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:06.158 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T09:42:06.274 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T09:42:06.408 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:42:06.408 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:06.409 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:06.409 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:42:06.409 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T09:42:06.438 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:06.438 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T09:42:06.439 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T09:42:06.440 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:06.440 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T09:42:06.591 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T09:42:06.685 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T09:42:06.809 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T09:42:06.827 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:06.926 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T09:42:07.037 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T09:42:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:07.060 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T09:42:07.061 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:42:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T09:42:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T09:42:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:07.062 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:42:07.062 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:07.062 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T09:42:07.063 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:07.063 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 316 k 2019-12-09T09:42:07.063 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 1.1 M 2019-12-09T09:42:07.063 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:42:07.185 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T09:42:07.286 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T09:42:07.407 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T09:42:07.636 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:Total 549 kB/s | 316 kB 00:00 2019-12-09T09:42:07.644 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:42:07.650 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:42:07.728 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:42:07.728 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:42:07.947 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T09:42:07.947 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:07.947 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:42:07.947 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:07.947 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T09:42:07.948 INFO:teuthology.orchestra.run.smithi161.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T09:42:07.949 INFO:teuthology.orchestra.run.smithi161.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T09:42:07.949 INFO:teuthology.orchestra.run.smithi161.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T09:42:07.949 INFO:teuthology.orchestra.run.smithi161.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T09:42:07.949 INFO:teuthology.orchestra.run.smithi161.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T09:42:07.949 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:07.949 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:42:08.074 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-ssh 2019-12-09T09:42:08.329 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:08.374 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:08.375 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:08.376 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:08.376 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:08.376 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:08.583 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T09:42:08.884 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T09:42:09.039 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:09.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:42:09.234 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T09:42:09.343 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T09:42:09.389 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:42:09.389 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:09.390 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:09.392 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:42:09.422 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:09.422 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T09:42:09.423 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T09:42:09.424 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:09.424 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T09:42:09.787 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:42:09.787 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:09.787 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:42:09.788 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:09.788 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:09.788 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:42:09.788 INFO:teuthology.orchestra.run.smithi059.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T09:42:09.788 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:09.788 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:42:09.790 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:09.924 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-fuse 2019-12-09T09:42:10.019 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:10.020 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:42:10.020 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:10.020 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T09:42:10.021 INFO:teuthology.orchestra.run.smithi161.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 316 k 2019-12-09T09:42:10.022 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 1.1 M 2019-12-09T09:42:10.023 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:42:10.177 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:10.222 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:10.223 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:10.224 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:10.224 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:10.224 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:10.598 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:10.598 INFO:teuthology.orchestra.run.smithi161.stdout:Total 548 kB/s | 316 kB 00:00 2019-12-09T09:42:10.602 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:42:10.611 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:42:10.693 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:42:10.693 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:42:10.897 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:11.249 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:42:11.249 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:11.250 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:11.261 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:42:11.298 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:42:11.298 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:11.298 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:42:11.298 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:11.298 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:11.299 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:11.300 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:11.300 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:11.300 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:42:11.307 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T09:42:11.307 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T09:42:11.483 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T09:42:11.693 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:11.805 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T09:42:11.925 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:11.926 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:42:11.926 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T09:42:11.927 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:42:11.928 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:11.929 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:42:11.929 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:11.929 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 10 M 2019-12-09T09:42:11.930 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 49 M 2019-12-09T09:42:11.930 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:42:12.106 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:42:12.227 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T09:42:12.298 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:12.299 INFO:teuthology.orchestra.run.smithi059.stdout:Total 27 MB/s | 10 MB 00:00 2019-12-09T09:42:12.319 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:42:12.346 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:42:12.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T09:42:12.476 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:42:12.477 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:42:12.887 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:42:12.888 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:12.888 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:42:12.888 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:12.888 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:12.888 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:42:12.888 INFO:teuthology.orchestra.run.smithi161.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T09:42:12.889 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:12.889 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:42:13.015 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-fuse 2019-12-09T09:42:13.235 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:13.273 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:13.274 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:13.275 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:13.275 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:13.275 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:13.913 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:14.240 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:42:14.241 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:14.241 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:14.251 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:42:14.290 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:42:14.290 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:14.291 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:42:14.291 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:14.291 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:14.291 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:14.291 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:42:14.291 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:14.292 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:42:14.295 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T09:42:14.295 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T09:42:14.658 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:14.817 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T09:42:14.883 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:14.884 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:42:14.885 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:14.885 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:14.885 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:42:14.885 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:42:14.885 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:14.885 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:42:14.886 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T09:42:14.886 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:42:14.886 INFO:teuthology.orchestra.run.smithi161.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T09:42:14.886 INFO:teuthology.orchestra.run.smithi161.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T09:42:14.886 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T09:42:14.886 INFO:teuthology.orchestra.run.smithi161.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T09:42:14.887 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:14.887 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:42:14.887 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:14.887 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:42:14.887 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:14.887 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 10 M 2019-12-09T09:42:14.888 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 49 M 2019-12-09T09:42:14.888 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:42:15.368 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T09:42:15.559 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:15.559 INFO:teuthology.orchestra.run.smithi161.stdout:Total 15 MB/s | 10 MB 00:00 2019-12-09T09:42:15.580 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:42:15.609 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:42:15.714 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:42:15.715 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:42:16.275 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T09:42:16.756 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T09:42:17.014 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:42:17.129 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:42:17.238 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T09:42:17.380 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T09:42:17.497 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T09:42:17.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T09:42:17.942 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:17.942 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:42:17.942 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:17.942 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:17.942 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:42:17.942 INFO:teuthology.orchestra.run.smithi059.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T09:42:17.943 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T09:42:17.943 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:17.943 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:42:18.012 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T09:42:18.067 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install libcephfs2 2019-12-09T09:42:18.253 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:18.555 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T09:42:18.942 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:19.281 INFO:teuthology.orchestra.run.smithi059.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:42:19.282 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do 2019-12-09T09:42:19.343 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install libcephfs-devel 2019-12-09T09:42:19.500 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T09:42:19.603 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:19.649 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:19.650 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:19.651 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:19.651 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:19.651 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:19.979 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T09:42:20.281 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:42:20.334 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:20.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:42:20.480 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T09:42:20.614 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T09:42:20.686 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:42:20.687 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:20.687 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:20.688 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:42:20.715 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T09:42:20.719 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:20.719 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:21.111 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:21.166 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T09:42:21.166 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:21.167 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:42:21.167 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:21.167 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:21.167 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:42:21.167 INFO:teuthology.orchestra.run.smithi161.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T09:42:21.167 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T09:42:21.168 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:21.168 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:42:21.299 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install libcephfs2 2019-12-09T09:42:21.346 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:21.347 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:42:21.347 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:21.347 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:21.348 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:42:21.348 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:42:21.348 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:21.348 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:42:21.348 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:42:21.349 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:42:21.349 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T09:42:21.349 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:21.349 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:42:21.349 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:21.350 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T09:42:21.350 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:21.350 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 170 k 2019-12-09T09:42:21.350 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 679 k 2019-12-09T09:42:21.351 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:42:21.532 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:21.572 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:21.574 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:21.574 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:21.574 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:21.574 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:21.729 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:21.729 INFO:teuthology.orchestra.run.smithi059.stdout:Total 445 kB/s | 170 kB 00:00 2019-12-09T09:42:21.732 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:42:21.745 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:42:21.748 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:42:21.749 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:42:22.227 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:22.328 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:42:22.558 INFO:teuthology.orchestra.run.smithi161.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:42:22.558 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do 2019-12-09T09:42:22.571 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:42:22.620 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install libcephfs-devel 2019-12-09T09:42:22.720 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:42:22.797 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:22.832 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:22.833 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:22.834 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:22.834 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:22.834 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:23.205 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:42:23.205 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:23.205 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:42:23.205 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:23.206 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:23.206 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:42:23.206 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:23.206 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:23.206 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:42:23.340 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install librados2 2019-12-09T09:42:23.429 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:23.604 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:23.650 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:23.651 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:23.652 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:23.652 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:23.652 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:23.754 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:42:23.755 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:23.755 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:23.755 INFO:teuthology.orchestra.run.smithi161.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:42:23.786 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:23.786 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:24.156 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:24.332 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:24.385 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:24.385 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:42:24.386 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:24.386 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:24.386 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:42:24.386 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:42:24.386 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:24.386 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout:Installing for dependencies: 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:24.387 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T09:42:24.388 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:24.388 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 170 k 2019-12-09T09:42:24.388 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 679 k 2019-12-09T09:42:24.388 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:42:24.683 INFO:teuthology.orchestra.run.smithi059.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:42:24.683 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do 2019-12-09T09:42:24.737 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:42:24.737 INFO:teuthology.orchestra.run.smithi161.stdout:Total 484 kB/s | 170 kB 00:00 2019-12-09T09:42:24.740 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:42:24.744 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install librbd1 2019-12-09T09:42:24.752 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:42:24.754 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:42:24.755 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:42:24.964 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:25.005 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:25.005 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:25.006 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:25.006 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:25.007 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:25.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:42:25.636 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:25.679 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:42:25.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:42:25.977 INFO:teuthology.orchestra.run.smithi059.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:42:25.977 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do 2019-12-09T09:42:26.038 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install python-ceph 2019-12-09T09:42:26.269 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:42:26.269 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout:Dependency Installed: 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:26.270 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:42:26.288 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:26.337 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:26.337 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:26.337 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:26.338 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:26.389 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install librados2 2019-12-09T09:42:26.610 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:26.650 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:26.651 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:26.652 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:26.652 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:26.652 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:27.019 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:27.280 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:27.373 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:42:27.374 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:27.374 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:27.606 INFO:teuthology.orchestra.run.smithi161.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:42:27.607 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do 2019-12-09T09:42:27.668 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install librbd1 2019-12-09T09:42:27.763 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:27.922 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:27.955 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:27.956 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:42:27.956 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout: Size 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T09:42:27.957 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 2.2 k 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 0 2019-12-09T09:42:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:42:27.966 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:27.967 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:27.968 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:27.968 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:27.969 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:28.221 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:42:28.224 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:42:28.225 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:42:28.226 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:42:28.629 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:28.734 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:28.965 INFO:teuthology.orchestra.run.smithi161.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:42:28.966 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do 2019-12-09T09:42:29.027 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install python-ceph 2019-12-09T09:42:29.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:29.218 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:29.218 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:42:29.218 INFO:teuthology.orchestra.run.smithi059.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:29.218 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:29.219 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:42:29.276 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:29.321 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:29.322 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:29.322 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:29.322 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:29.323 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:29.345 INFO:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-fuse 2019-12-09T09:42:29.583 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:29.629 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:29.630 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:29.630 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:29.630 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:29.631 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:29.979 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:30.316 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:30.319 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:42:30.319 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:30.319 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:30.671 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:42:30.671 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:42:30.671 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:30.690 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:30.877 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:30.892 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:42:30.892 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:30.892 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:30.893 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository 2019-12-09T09:42:30.893 INFO:teuthology.orchestra.run.smithi161.stdout: Size 2019-12-09T09:42:30.893 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:30.893 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:42:30.893 INFO:teuthology.orchestra.run.smithi161.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T09:42:30.893 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:30.899 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:42:30.900 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:30.900 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2019-12-09T09:42:30.900 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:30.900 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 2.2 k 2019-12-09T09:42:30.900 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 0 2019-12-09T09:42:30.900 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:42:31.069 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:31.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:31.261 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:42:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:31.262 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:31.262 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:42:31.262 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:31.262 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:31.263 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 76 k 2019-12-09T09:42:31.264 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 200 k 2019-12-09T09:42:31.264 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:42:31.360 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:42:31.361 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:42:31.363 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:42:31.364 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:42:31.789 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:42:31.804 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:42:31.805 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:42:31.806 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:42:32.033 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:32.592 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:32.592 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:32.593 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:42:32.593 INFO:teuthology.orchestra.run.smithi161.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:32.593 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:32.593 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:42:32.715 INFO:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-fuse 2019-12-09T09:42:32.864 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:32.864 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:32.864 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:42:32.864 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:32.864 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:42:32.865 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:42:32.899 INFO:teuthology.orchestra.run.smithi161.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:42:32.937 INFO:teuthology.orchestra.run.smithi161.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:42:32.938 INFO:teuthology.orchestra.run.smithi161.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:42:32.938 INFO:teuthology.orchestra.run.smithi161.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:42:32.939 INFO:teuthology.orchestra.run.smithi161.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:42:32.939 INFO:teuthology.orchestra.run.smithi161.stdout: * updates: mirror.cogentco.com 2019-12-09T09:42:32.984 DEBUG:teuthology.parallel:result is None 2019-12-09T09:42:33.559 INFO:teuthology.orchestra.run.smithi161.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:42:33.885 INFO:teuthology.orchestra.run.smithi161.stdout:Resolving Dependencies 2019-12-09T09:42:33.886 INFO:teuthology.orchestra.run.smithi161.stdout:--> Running transaction check 2019-12-09T09:42:33.886 INFO:teuthology.orchestra.run.smithi161.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:42:34.261 INFO:teuthology.orchestra.run.smithi161.stdout:--> Finished Dependency Resolution 2019-12-09T09:42:34.446 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:34.447 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies Resolved 2019-12-09T09:42:34.447 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:34.448 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:34.448 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2019-12-09T09:42:34.448 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:34.448 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2019-12-09T09:42:34.448 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T09:42:34.448 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 76 k 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 200 k 2019-12-09T09:42:34.449 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading packages: 2019-12-09T09:42:34.756 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2019-12-09T09:42:34.776 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2019-12-09T09:42:34.777 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded 2019-12-09T09:42:34.778 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2019-12-09T09:42:35.370 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:35.905 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:42:35.906 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:35.906 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2019-12-09T09:42:35.906 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:35.906 INFO:teuthology.orchestra.run.smithi161.stdout: 2019-12-09T09:42:35.907 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2019-12-09T09:42:36.025 DEBUG:teuthology.parallel:result is None 2019-12-09T09:42:36.026 INFO:teuthology.orchestra.run.smithi059:> sudo lsb_release -is 2019-12-09T09:42:36.188 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS 2019-12-09T09:42:36.191 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:42:36.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:42:36.365 INFO:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T09:42:36.511 INFO:teuthology.orchestra.run.smithi059.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:36.512 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T09:42:36.512 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T09:42:36.512 INFO:teuthology.orchestra.run.smithi161:> sudo lsb_release -is 2019-12-09T09:42:36.664 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS 2019-12-09T09:42:36.666 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:42:36.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:42:36.829 INFO:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T09:42:36.964 INFO:teuthology.orchestra.run.smithi161.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T09:42:36.964 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T09:42:36.965 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T09:42:36.965 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-12-09T09:42:36.965 INFO:teuthology.orchestra.run.smithi059:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T09:42:37.091 INFO:teuthology.orchestra.run.smithi161:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T09:42:37.214 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-12-09T09:42:37.215 INFO:teuthology.orchestra.run.smithi059:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T09:42:37.340 INFO:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T09:42:37.451 INFO:teuthology.orchestra.run.smithi161:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T09:42:37.567 INFO:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T09:42:37.663 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-12-09T09:42:37.663 INFO:teuthology.orchestra.run.smithi059:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T09:42:37.786 INFO:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T09:42:37.903 INFO:teuthology.orchestra.run.smithi161:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T09:42:38.022 INFO:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T09:42:38.144 INFO:teuthology.run_tasks:Running task ceph... 2019-12-09T09:42:38.226 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-12-09T09:42:38.226 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2019-12-09T09:42:38.231 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2019-12-09T09:42:38.353 INFO:tasks.ceph:Disabling ceph logrotate... 2019-12-09T09:42:38.353 INFO:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T09:42:38.360 INFO:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T09:42:38.470 INFO:tasks.ceph:Creating extra log directories... 2019-12-09T09:42:38.470 INFO:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T09:42:38.474 INFO:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T09:42:38.578 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-12-09T09:42:38.579 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd fast shutdown': False, 'debug osd': 25, 'osd objectstore': 'filestore', 'osd sloppy crc': True, 'debug filestore': 20, 'osd shutdown pgref assert': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-12-09T09:42:38.579 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.13080', 'owner': 'scheduled_ofriedma@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi059.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6AGl/ohYvVLjyWTLFhWYFlZ+ZYWOB5t3S49q+vH9Mjic+B4zvNAQCGAZYRxb3T1qeLcX0nEns8q+Cu5vuSMMzUYuEU4y8Y/Rw/qIrqcFozyMiozUP68cGa2aqPR0MDwJRjCsi9o/ZFTv8mcX9ki1vio+dkOS1azdeZuriM27jEOVSS7torHGoOeymFDuHIvPa+LSZOGIRwvIfkFqCtsO3m4yjh0VcRzTu/QkIS4XEQRnsmDUpxC18LyuvxQ2WnLqt0iWFe4GzS+4UDXNuO25eo0FTlcPPwCmI927A+9KTCb3GwOTWW31nd2bIc85cyUqt2Vk+C7n1uGBIP4W4DeiL', 'smithi161.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKKvsQpzhSuyM8U0q9CYbJsswotvPUE+t2JZkTvdffgHWi1jrLr6eF/BfCXio+oj+x3/o/rMvt/ABpUKakneYi8dioxoa+vCX0fPWpRs6mG0OvuZijylJLyOw2h8Oq2FgljuFJev584sCWJAnm5Br35IKwbM3RcagvELsxjp8XHQJz4YdRq/o5kylw0vvslYHRwooXOFr/+Nqzx1HTOc4nhSPO23R/aoTOiBlzCf4HEES/uMPg58PvzqLq6NY7Aq4Hj6SF9wl2/xleNDpgFSU9GgWKyrdMRKXVhEorvvkzLIXM8bVmfrmagij8+myNiVRZ8KoBjgXfw5N6i58bmw4X'}, 'job_id': '4584247', 'tube': 'smithi', 'suite_branch': 'wip-ofriedma-dynamic-disable', 'priority': 60, 'first_in_suite': False, 'branch': 'wip-ofriedma-dynamic-disable', 'suite': 'rgw', 'email': 'ofriedma@redhat.com', 'archive_path': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584247', 'description': 'rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'name': 'ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'os_type': 'centos', 'overrides': {'s3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'filestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd sloppy crc': True, 'osd objectstore': 'filestore'}}}, 'rgw': {'frontend': 'beast', 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'ec-data-pool': True, 'client.1': {'ssl certificate': 'rgw.client.1'}, 'client.0': {'ssl certificate': 'rgw.client.0'}, 'storage classes': 'LUKEWARM, FROZEN', 'compression type': 'random'}, 'workunit': {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'branch': 'wip-ofriedma-dynamic-disable'}, 'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd fast shutdown': False, 'debug osd': 25, 'osd objectstore': 'filestore', 'osd sloppy crc': True, 'debug filestore': 20, 'osd shutdown pgref assert': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}, 'openssl_keys': {'rgw.client.1': {'ca': 'root', 'client': 'client.1', 'embed-key': True}, 'root': {'key-type': 'rsa:4096', 'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0', 'client.1']}, 'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, 'admin_socket': {'branch': 'wip-ofriedma-dynamic-disable'}}} 2019-12-09T09:42:38.580 INFO:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T09:42:38.583 INFO:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T09:42:38.647 INFO:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T09:42:38.650 INFO:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T09:42:38.822 DEBUG:teuthology.orchestra.remote:smithi059:/scratch_devs is 72B 2019-12-09T09:42:38.897 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:42:38.897 INFO:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T09:42:38.966 INFO:teuthology.orchestra.run.smithi059.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T09:42:38.966 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:38.966 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 104757 Links: 1 2019-12-09T09:42:38.966 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:38.966 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:38.966 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2019-12-09 09:41:26.344171032 +0000 2019-12-09T09:42:38.967 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2019-12-09 09:39:42.278714649 +0000 2019-12-09T09:42:38.967 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2019-12-09 09:39:42.278714649 +0000 2019-12-09T09:42:38.967 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2019-12-09T09:42:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2019-12-09T09:42:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2019-12-09T09:42:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes (512 B) copied, 0.000457291 s, 1.1 MB/s 2019-12-09T09:42:38.992 INFO:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T09:42:39.120 INFO:teuthology.orchestra.run.smithi059.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T09:42:39.120 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.120 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 97897 Links: 1 2019-12-09T09:42:39.121 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.121 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.121 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2019-12-09 09:41:26.345171032 +0000 2019-12-09T09:42:39.121 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2019-12-09 09:39:42.756706343 +0000 2019-12-09T09:42:39.121 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2019-12-09 09:39:42.756706343 +0000 2019-12-09T09:42:39.122 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2019-12-09T09:42:39.152 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2019-12-09T09:42:39.152 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2019-12-09T09:42:39.153 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes (512 B) copied, 0.000494213 s, 1.0 MB/s 2019-12-09T09:42:39.157 INFO:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T09:42:39.232 INFO:teuthology.orchestra.run.smithi059.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T09:42:39.233 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.233 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 108835 Links: 1 2019-12-09T09:42:39.233 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.233 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.233 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2019-12-09 09:41:26.345171032 +0000 2019-12-09T09:42:39.234 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2019-12-09 09:39:43.256697655 +0000 2019-12-09T09:42:39.234 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2019-12-09 09:39:43.256697655 +0000 2019-12-09T09:42:39.234 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2019-12-09T09:42:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2019-12-09T09:42:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2019-12-09T09:42:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes (512 B) copied, 0.000419045 s, 1.2 MB/s 2019-12-09T09:42:39.265 INFO:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T09:42:39.330 INFO:teuthology.orchestra.run.smithi059.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T09:42:39.330 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.330 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 97929 Links: 1 2019-12-09T09:42:39.331 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.331 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.331 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2019-12-09 09:41:26.345171032 +0000 2019-12-09T09:42:39.331 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2019-12-09 09:39:43.727689470 +0000 2019-12-09T09:42:39.331 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2019-12-09 09:39:43.727689470 +0000 2019-12-09T09:42:39.331 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2019-12-09T09:42:39.354 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2019-12-09T09:42:39.354 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2019-12-09T09:42:39.354 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes (512 B) copied, 0.000456451 s, 1.1 MB/s 2019-12-09T09:42:39.360 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T09:42:39.360 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:42:39.360 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T09:42:39.360 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:39.458 DEBUG:teuthology.orchestra.remote:smithi161:/scratch_devs is 72B 2019-12-09T09:42:39.536 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:42:39.536 INFO:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T09:42:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T09:42:39.603 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.603 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 107661 Links: 1 2019-12-09T09:42:39.604 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.604 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.604 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2019-12-09 09:41:28.596859382 +0000 2019-12-09T09:42:39.604 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2019-12-09 09:39:38.369217587 +0000 2019-12-09T09:42:39.604 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2019-12-09 09:39:38.369217587 +0000 2019-12-09T09:42:39.605 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2019-12-09T09:42:39.628 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2019-12-09T09:42:39.628 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2019-12-09T09:42:39.628 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes (512 B) copied, 0.000345379 s, 1.5 MB/s 2019-12-09T09:42:39.632 INFO:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T09:42:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T09:42:39.688 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 91945 Links: 1 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2019-12-09 09:41:28.596859382 +0000 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2019-12-09 09:39:38.870209558 +0000 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2019-12-09 09:39:38.870209558 +0000 2019-12-09T09:42:39.689 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2019-12-09T09:42:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2019-12-09T09:42:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2019-12-09T09:42:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes (512 B) copied, 0.000295467 s, 1.7 MB/s 2019-12-09T09:42:39.713 INFO:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T09:42:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T09:42:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.763 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 108725 Links: 1 2019-12-09T09:42:39.763 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.763 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.763 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2019-12-09 09:41:28.596859382 +0000 2019-12-09T09:42:39.764 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2019-12-09 09:39:39.354201800 +0000 2019-12-09T09:42:39.764 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2019-12-09 09:39:39.354201800 +0000 2019-12-09T09:42:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2019-12-09T09:42:39.780 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2019-12-09T09:42:39.780 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2019-12-09T09:42:39.780 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes (512 B) copied, 0.000299845 s, 1.7 MB/s 2019-12-09T09:42:39.783 INFO:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T09:42:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T09:42:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 110631 Links: 1 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2019-12-09 09:41:28.596859382 +0000 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2019-12-09 09:39:39.826194236 +0000 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2019-12-09 09:39:39.826194236 +0000 2019-12-09T09:42:39.830 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2019-12-09T09:42:39.845 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2019-12-09T09:42:39.845 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2019-12-09T09:42:39.845 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes (512 B) copied, 0.000295309 s, 1.7 MB/s 2019-12-09T09:42:39.848 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T09:42:39.848 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:42:39.848 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T09:42:39.848 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:39.848 INFO:tasks.ceph:Generating config... 2019-12-09T09:42:39.850 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-12-09T09:42:39.850 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-12-09T09:42:39.851 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-12-09T09:42:39.851 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-12-09T09:42:39.851 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-12-09T09:42:39.851 INFO:tasks.ceph:[osd] debug ms = 1 2019-12-09T09:42:39.851 INFO:tasks.ceph:[osd] debug journal = 20 2019-12-09T09:42:39.851 INFO:tasks.ceph:[osd] osd fast shutdown = False 2019-12-09T09:42:39.851 INFO:tasks.ceph:[osd] debug osd = 25 2019-12-09T09:42:39.852 INFO:tasks.ceph:[osd] osd objectstore = filestore 2019-12-09T09:42:39.852 INFO:tasks.ceph:[osd] osd sloppy crc = True 2019-12-09T09:42:39.852 INFO:tasks.ceph:[osd] debug filestore = 20 2019-12-09T09:42:39.852 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-12-09T09:42:39.855 INFO:tasks.ceph:[mon] debug mon = 20 2019-12-09T09:42:39.855 INFO:tasks.ceph:[mon] debug paxos = 20 2019-12-09T09:42:39.855 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-12-09T09:42:39.855 INFO:tasks.ceph:[mon] debug ms = 1 2019-12-09T09:42:39.855 INFO:tasks.ceph:[client] debug rgw = 20 2019-12-09T09:42:39.855 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-12-09T09:42:39.855 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-12-09T09:42:39.856 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-12-09T09:42:39.856 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-12-09T09:42:39.856 INFO:tasks.ceph:Setting up mon.a... 2019-12-09T09:42:39.856 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-12-09T09:42:40.048 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.keyring 2019-12-09T09:42:40.053 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-12-09T09:42:40.208 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:42:40.323 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.59'), ('mon.c', '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]'), ('mon.b', '172.21.15.161')] 2019-12-09T09:42:40.323 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.59 --addv c '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]' --add b 172.21.15.161 --print /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:40.468 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:40.469 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: generated fsid 9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf 2019-12-09T09:42:40.469 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 0 2019-12-09T09:42:40.469 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf 2019-12-09T09:42:40.469 INFO:teuthology.orchestra.run.smithi059.stdout:last_changed 2019-12-09T09:42:40.468862+0000 2019-12-09T09:42:40.470 INFO:teuthology.orchestra.run.smithi059.stdout:created 2019-12-09T09:42:40.468862+0000 2019-12-09T09:42:40.470 INFO:teuthology.orchestra.run.smithi059.stdout:min_mon_release 0 (unknown) 2019-12-09T09:42:40.470 INFO:teuthology.orchestra.run.smithi059.stdout:0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2019-12-09T09:42:40.470 INFO:teuthology.orchestra.run.smithi059.stdout:1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2019-12-09T09:42:40.471 INFO:teuthology.orchestra.run.smithi059.stdout:2: [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] mon.c 2019-12-09T09:42:40.471 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-12-09T09:42:40.474 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf... 2019-12-09T09:42:40.475 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T09:42:40.488 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T09:42:40.705 INFO:tasks.ceph:Creating admin key on mon.a... 2019-12-09T09:42:40.705 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-12-09T09:42:40.842 INFO:tasks.ceph:Copying monmap to all nodes... 2019-12-09T09:42:40.929 DEBUG:teuthology.orchestra.remote:smithi059:/etc/ceph/ceph.keyring is 206B 2019-12-09T09:42:41.108 DEBUG:teuthology.orchestra.remote:smithi059:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-12-09T09:42:41.188 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:42:41.189 INFO:teuthology.orchestra.run.smithi059:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:42:41.338 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:41.404 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:42:41.405 INFO:teuthology.orchestra.run.smithi161:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:42:41.549 INFO:teuthology.orchestra.run.smithi161:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:41.621 INFO:tasks.ceph:Setting up mon nodes... 2019-12-09T09:42:41.621 INFO:tasks.ceph:Setting up mgr nodes... 2019-12-09T09:42:41.622 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T09:42:41.790 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T09:42:41.801 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T09:42:42.019 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T09:42:42.030 INFO:tasks.ceph:Setting up mds nodes... 2019-12-09T09:42:42.031 INFO:tasks.ceph_client:Setting up client nodes... 2019-12-09T09:42:42.031 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-12-09T09:42:42.184 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-12-09T09:42:42.219 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-12-09T09:42:42.368 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-12-09T09:42:42.401 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-12-09T09:42:42.401 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}} 2019-12-09T09:42:42.402 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-12-09T09:42:42.523 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:42.523 INFO:tasks.ceph:{} 2019-12-09T09:42:42.523 INFO:tasks.ceph:osd.0 2019-12-09T09:42:42.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:42:42.524 INFO:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T09:42:43.127 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:43.127 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:43.128 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:43.128 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:43.128 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:43.129 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2019-12-09T09:42:43.129 INFO:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-12-09T09:42:43.312 INFO:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-12-09T09:42:43.438 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-12-09T09:42:43.541 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:43.541 INFO:tasks.ceph:{} 2019-12-09T09:42:43.542 INFO:tasks.ceph:osd.1 2019-12-09T09:42:43.542 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:42:43.542 INFO:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T09:42:44.138 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:44.138 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:44.138 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:44.138 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:44.139 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:44.139 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:44.139 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:44.140 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2019-12-09T09:42:44.141 INFO:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-12-09T09:42:44.287 INFO:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-12-09T09:42:44.380 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-12-09T09:42:44.462 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:44.462 INFO:tasks.ceph:{} 2019-12-09T09:42:44.462 INFO:tasks.ceph:osd.2 2019-12-09T09:42:44.462 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:42:44.462 INFO:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T09:42:45.058 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:45.059 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:45.060 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:45.061 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2019-12-09T09:42:45.061 INFO:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-12-09T09:42:45.254 INFO:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-12-09T09:42:45.382 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-12-09T09:42:45.489 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:45.489 INFO:tasks.ceph:{} 2019-12-09T09:42:45.490 INFO:tasks.ceph:osd.3 2019-12-09T09:42:45.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T09:42:45.490 INFO:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T09:42:46.083 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:46.083 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:46.084 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:46.084 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:46.084 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:46.084 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:46.084 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:46.084 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:46.085 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:46.086 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2019-12-09T09:42:46.087 INFO:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-12-09T09:42:46.245 INFO:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-12-09T09:42:46.366 INFO:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.504+0000 7f6f610b8b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-12-09T09:42:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.505+0000 7f6f610b8b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-12-09T09:42:46.548 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.547+0000 7f6f610b8b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:46.548 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.548+0000 7f6f610b8b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.806+0000 7f6f610b8b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.807+0000 7f6f610b8b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:46.807+0000 7f6f610b8b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:46.836 INFO:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-12-09T09:42:46.951 INFO:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:47.095 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.095+0000 7f870442cb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-12-09T09:42:47.098 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.098+0000 7f870442cb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-12-09T09:42:47.130 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.130+0000 7f870442cb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:47.130 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.130+0000 7f870442cb40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:47.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.352+0000 7f870442cb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:47.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.352+0000 7f870442cb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:47.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.352+0000 7f870442cb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:47.365 INFO:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-12-09T09:42:47.466 INFO:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:47.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.599+0000 7f409493ab40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-12-09T09:42:47.600 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.600+0000 7f409493ab40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-12-09T09:42:47.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.614+0000 7f409493ab40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:47.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.614+0000 7f409493ab40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:47.752 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.752+0000 7f409493ab40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:47.752 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.752+0000 7f409493ab40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:47.752 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:47.752+0000 7f409493ab40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:47.765 INFO:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-12-09T09:42:47.870 INFO:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:48.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.011+0000 7fd9e62dcb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-12-09T09:42:48.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.012+0000 7fd9e62dcb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-12-09T09:42:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.026+0000 7fd9e62dcb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.026+0000 7fd9e62dcb40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:48.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.155+0000 7fd9e62dcb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:48.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.155+0000 7fd9e62dcb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:48.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:42:48.155+0000 7fd9e62dcb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:48.167 INFO:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-12-09T09:42:48.254 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-12-09T09:42:48.373 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:48.373 INFO:tasks.ceph:{} 2019-12-09T09:42:48.373 INFO:tasks.ceph:osd.4 2019-12-09T09:42:48.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:42:48.374 INFO:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T09:42:48.984 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:48.984 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:48.984 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:48.984 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:48.985 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:48.985 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:48.985 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:48.985 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:48.985 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:48.988 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2019-12-09T09:42:48.988 INFO:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-12-09T09:42:49.175 INFO:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-12-09T09:42:49.287 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-12-09T09:42:49.390 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:49.390 INFO:tasks.ceph:{} 2019-12-09T09:42:49.391 INFO:tasks.ceph:osd.5 2019-12-09T09:42:49.391 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:42:49.391 INFO:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T09:42:49.964 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:49.964 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:49.964 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:49.964 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:49.964 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:49.964 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:49.965 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:49.965 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:49.965 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:49.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2019-12-09T09:42:49.967 INFO:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-12-09T09:42:50.095 INFO:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-12-09T09:42:50.184 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-12-09T09:42:50.285 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:50.285 INFO:tasks.ceph:{} 2019-12-09T09:42:50.285 INFO:tasks.ceph:osd.6 2019-12-09T09:42:50.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:42:50.286 INFO:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T09:42:50.893 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:50.894 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:50.894 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:50.894 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:50.894 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:50.895 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:50.895 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:50.897 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2019-12-09T09:42:50.898 INFO:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-12-09T09:42:51.077 INFO:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-12-09T09:42:51.192 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-12-09T09:42:51.285 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:42:51.285 INFO:tasks.ceph:{} 2019-12-09T09:42:51.285 INFO:tasks.ceph:osd.7 2019-12-09T09:42:51.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T09:42:51.286 INFO:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T09:42:51.834 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:42:51.834 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:42:51.835 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:42:51.836 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2019-12-09T09:42:51.837 INFO:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-12-09T09:42:51.976 INFO:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-12-09T09:42:52.063 INFO:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.190+0000 7f78f0fadb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-12-09T09:42:52.192 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.192+0000 7f78f0fadb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-12-09T09:42:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.219+0000 7f78f0fadb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.219+0000 7f78f0fadb40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:52.476 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.475+0000 7f78f0fadb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:52.477 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.476+0000 7f78f0fadb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:52.477 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.476+0000 7f78f0fadb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:52.507 INFO:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-12-09T09:42:52.618 INFO:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:52.775 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.776+0000 7f4f27183b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-12-09T09:42:52.776 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.777+0000 7f4f27183b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-12-09T09:42:52.817 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.818+0000 7f4f27183b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:52.817 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:52.818+0000 7f4f27183b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:53.076 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.077+0000 7f4f27183b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:53.079 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.080+0000 7f4f27183b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:53.079 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.080+0000 7f4f27183b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:53.104 INFO:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-12-09T09:42:53.212 INFO:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:53.360 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.361+0000 7ff1eb122b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-12-09T09:42:53.361 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.362+0000 7ff1eb122b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-12-09T09:42:53.377 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.377+0000 7ff1eb122b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:53.377 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.378+0000 7ff1eb122b40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:53.521 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.522+0000 7ff1eb122b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:53.521 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.522+0000 7ff1eb122b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:53.522 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.522+0000 7ff1eb122b40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:53.533 INFO:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-12-09T09:42:53.638 INFO:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:42:53.772 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.772+0000 7f0c31c3eb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-12-09T09:42:53.772 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.773+0000 7f0c31c3eb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-12-09T09:42:53.785 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.786+0000 7f0c31c3eb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:53.786 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.786+0000 7f0c31c3eb40 -1 journal FileJournal::_open_file : zeroing journal 2019-12-09T09:42:53.929 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.927+0000 7f0c31c3eb40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:42:53.929 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.927+0000 7f0c31c3eb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:53.929 INFO:teuthology.orchestra.run.smithi161.stderr:2019-12-09T09:42:53.927+0000 7f0c31c3eb40 -1 journal do_read_entry(4096): bad header magic 2019-12-09T09:42:53.939 INFO:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-12-09T09:42:54.046 INFO:tasks.ceph:Reading keys from all nodes... 2019-12-09T09:42:54.046 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:42:54.139 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.RdxZ5jUi1L 2019-12-09T09:42:54.139 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.RdxZ5jUi1L 2019-12-09T09:42:54.253 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.RdxZ5jUi1L 2019-12-09T09:42:54.452 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.RdxZ5jUi1L is 56B 2019-12-09T09:42:54.553 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.RdxZ5jUi1L 2019-12-09T09:42:54.642 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:42:54.718 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.0635O2gIly 2019-12-09T09:42:54.719 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.0635O2gIly 2019-12-09T09:42:54.827 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.0635O2gIly 2019-12-09T09:42:55.013 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.0635O2gIly is 56B 2019-12-09T09:42:55.095 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.0635O2gIly 2019-12-09T09:42:55.170 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:42:55.250 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.2UVmIdkKNH 2019-12-09T09:42:55.250 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.2UVmIdkKNH 2019-12-09T09:42:55.351 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.2UVmIdkKNH 2019-12-09T09:42:55.547 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.2UVmIdkKNH is 56B 2019-12-09T09:42:55.640 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.2UVmIdkKNH 2019-12-09T09:42:55.725 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:42:55.802 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.erSbfHMzEU 2019-12-09T09:42:55.803 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.erSbfHMzEU 2019-12-09T09:42:55.924 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.erSbfHMzEU 2019-12-09T09:42:56.115 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.erSbfHMzEU is 56B 2019-12-09T09:42:56.194 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.erSbfHMzEU 2019-12-09T09:42:56.270 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:42:56.402 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.EsrPTqHnaJ 2019-12-09T09:42:56.403 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.EsrPTqHnaJ 2019-12-09T09:42:56.523 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.EsrPTqHnaJ 2019-12-09T09:42:56.732 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.EsrPTqHnaJ is 56B 2019-12-09T09:42:56.822 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.EsrPTqHnaJ 2019-12-09T09:42:56.893 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:42:56.983 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.pnaUyH36Xo 2019-12-09T09:42:56.984 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.pnaUyH36Xo 2019-12-09T09:42:57.089 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.pnaUyH36Xo 2019-12-09T09:42:57.274 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.pnaUyH36Xo is 56B 2019-12-09T09:42:57.342 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.pnaUyH36Xo 2019-12-09T09:42:57.411 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:42:57.468 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.z6avrE8MGx 2019-12-09T09:42:57.468 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.z6avrE8MGx 2019-12-09T09:42:57.539 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.z6avrE8MGx 2019-12-09T09:42:57.659 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.z6avrE8MGx is 56B 2019-12-09T09:42:57.713 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.z6avrE8MGx 2019-12-09T09:42:57.780 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:42:57.894 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.tUi2tzvHaR 2019-12-09T09:42:57.895 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.tUi2tzvHaR 2019-12-09T09:42:57.998 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.tUi2tzvHaR 2019-12-09T09:42:58.193 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.tUi2tzvHaR is 56B 2019-12-09T09:42:58.271 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.tUi2tzvHaR 2019-12-09T09:42:58.338 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:42:58.397 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.m9npHYxMUY 2019-12-09T09:42:58.397 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.m9npHYxMUY 2019-12-09T09:42:58.473 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.m9npHYxMUY 2019-12-09T09:42:58.589 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.m9npHYxMUY is 56B 2019-12-09T09:42:58.647 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.m9npHYxMUY 2019-12-09T09:42:58.706 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:42:58.837 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.3AClPpGi65 2019-12-09T09:42:58.838 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.3AClPpGi65 2019-12-09T09:42:58.924 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.3AClPpGi65 2019-12-09T09:42:59.057 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.3AClPpGi65 is 56B 2019-12-09T09:42:59.114 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.3AClPpGi65 2019-12-09T09:42:59.279 DEBUG:teuthology.orchestra.remote:smithi059:/etc/ceph/ceph.client.0.keyring is 59B 2019-12-09T09:42:59.455 DEBUG:teuthology.orchestra.remote:smithi161:/etc/ceph/ceph.client.1.keyring is 59B 2019-12-09T09:42:59.578 INFO:tasks.ceph:Adding keys to all mons... 2019-12-09T09:42:59.578 INFO:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T09:42:59.582 INFO:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T09:42:59.692 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.y] 2019-12-09T09:42:59.692 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQARF+5dhTUlLxAAqK8jS6Gy/H2EHySogAFw9g== 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAWF+5dUpwUHhAAXDFXMnaMeC6gx7olw9LfQg== 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAXF+5dVyeyBRAAwrQdpeUy/u+/LIYbjvtT6A== 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAXF+5dDjfIIxAA8524QefGAYThnl4mPtfE4A== 2019-12-09T09:42:59.693 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAYF+5d3/64ABAA2RamXZN8g9EBS7ZHCFl1IA== 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQASF+5dhOlDARAARUGv5qdgdOQk6Uz0dR+9TA== 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAcF+5d1VtvCxAAMyFjjYIe3ux0Q+H8R2qQfg== 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2019-12-09T09:42:59.694 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAcF+5dmKBVLhAACzmKTDnkm74XB6wBaWWkbQ== 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.6] 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAdF+5dVsKdFRAAYOOjFbX5OjQDoO7lLxf3LA== 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.7] 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAdF+5dZREhLhAAwJyRfq/YKsFDu2lWmxS3xA== 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQASF+5dkLEBCxAAEBfzOspDV+bTJTt8FA04yA== 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout:[client.1] 2019-12-09T09:42:59.695 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQASF+5d3yQWFhAAXXZUyzuO5vCfMlDAa35Kow== 2019-12-09T09:42:59.698 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQARF+5dhTUlLxAAqK8jS6Gy/H2EHySogAFw9g== 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAWF+5dUpwUHhAAXDFXMnaMeC6gx7olw9LfQg== 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAXF+5dVyeyBRAAwrQdpeUy/u+/LIYbjvtT6A== 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2019-12-09T09:42:59.699 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAXF+5dDjfIIxAA8524QefGAYThnl4mPtfE4A== 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAYF+5d3/64ABAA2RamXZN8g9EBS7ZHCFl1IA== 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQASF+5dhOlDARAARUGv5qdgdOQk6Uz0dR+9TA== 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAcF+5d1VtvCxAAMyFjjYIe3ux0Q+H8R2qQfg== 2019-12-09T09:42:59.700 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAcF+5dmKBVLhAACzmKTDnkm74XB6wBaWWkbQ== 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAdF+5dVsKdFRAAYOOjFbX5OjQDoO7lLxf3LA== 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQAdF+5dZREhLhAAwJyRfq/YKsFDu2lWmxS3xA== 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2019-12-09T09:42:59.701 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQASF+5dkLEBCxAAEBfzOspDV+bTJTt8FA04yA== 2019-12-09T09:42:59.702 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2019-12-09T09:42:59.702 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQASF+5d3yQWFhAAXXZUyzuO5vCfMlDAa35Kow== 2019-12-09T09:42:59.703 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:42:59.706 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:42:59.872 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:42:59.876 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.047 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.053 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.226 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.230 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.396 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.399 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.564 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:43:00.568 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:43:00.739 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.743 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.915 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:00.918 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:01.089 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:01.093 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:01.258 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:01.262 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:43:01.423 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:43:01.427 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:43:01.604 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:43:01.682 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:43:01.862 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-12-09T09:43:01.863 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-12-09T09:43:01.978 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:43:02.158 INFO:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-12-09T09:43:02.264 INFO:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-12-09T09:43:02.372 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:43:02.556 INFO:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-12-09T09:43:02.666 INFO:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-12-09T09:43:02.787 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:43:02.942 INFO:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-12-09T09:43:03.051 INFO:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:43:03.058 INFO:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:43:03.146 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-12-09T09:43:03.146 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:43:03.146 INFO:tasks.ceph.mon.a:Restarting daemon 2019-12-09T09:43:03.147 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-12-09T09:43:03.175 INFO:tasks.ceph.mon.a:Started 2019-12-09T09:43:03.175 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:43:03.176 INFO:tasks.ceph.mon.c:Restarting daemon 2019-12-09T09:43:03.176 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-12-09T09:43:03.183 INFO:tasks.ceph.mon.c:Started 2019-12-09T09:43:03.183 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:43:03.183 INFO:tasks.ceph.mon.b:Restarting daemon 2019-12-09T09:43:03.184 INFO:teuthology.orchestra.run.smithi161:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-12-09T09:43:03.188 INFO:tasks.ceph.mon.b:Started 2019-12-09T09:43:03.188 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-12-09T09:43:03.188 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-12-09T09:43:03.188 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-12-09T09:43:03.192 INFO:tasks.ceph.mgr.y:Started 2019-12-09T09:43:03.192 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-12-09T09:43:03.192 INFO:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-12-09T09:43:03.195 INFO:tasks.ceph.mgr.x:Started 2019-12-09T09:43:03.195 INFO:tasks.ceph:Setting crush tunables to default 2019-12-09T09:43:03.195 INFO:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd crush tunables default 2019-12-09T09:43:07.365 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:00:03.967 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:07.452 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:00:04.074 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:07.459 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:00:04.077 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:20.017 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2019-12-09T09:43:20.033 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-12-09T09:43:20.033 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:43:20.097 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.PRc7tXBXzw 2019-12-09T09:43:20.097 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.PRc7tXBXzw 2019-12-09T09:43:20.167 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.PRc7tXBXzw 2019-12-09T09:43:20.281 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.PRc7tXBXzw is 37B 2019-12-09T09:43:20.332 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.PRc7tXBXzw 2019-12-09T09:43:20.379 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:43:20.482 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.ESL9pFknNS 2019-12-09T09:43:20.482 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.ESL9pFknNS 2019-12-09T09:43:20.545 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.ESL9pFknNS 2019-12-09T09:43:20.642 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.ESL9pFknNS is 37B 2019-12-09T09:43:20.701 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.ESL9pFknNS 2019-12-09T09:43:20.753 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:43:20.837 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.TVDmSXRA4E 2019-12-09T09:43:20.837 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.TVDmSXRA4E 2019-12-09T09:43:20.891 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.TVDmSXRA4E 2019-12-09T09:43:21.008 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.TVDmSXRA4E is 37B 2019-12-09T09:43:21.058 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.TVDmSXRA4E 2019-12-09T09:43:21.099 INFO:teuthology.orchestra.run.smithi059:> mktemp 2019-12-09T09:43:21.187 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.SwUF0ygPBv 2019-12-09T09:43:21.188 INFO:teuthology.orchestra.run.smithi059:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.SwUF0ygPBv 2019-12-09T09:43:21.241 INFO:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.SwUF0ygPBv 2019-12-09T09:43:21.347 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.SwUF0ygPBv is 37B 2019-12-09T09:43:21.401 INFO:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.SwUF0ygPBv 2019-12-09T09:43:21.441 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:43:21.526 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.YOxd9GUJn2 2019-12-09T09:43:21.526 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.YOxd9GUJn2 2019-12-09T09:43:21.628 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.YOxd9GUJn2 2019-12-09T09:43:21.770 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.YOxd9GUJn2 is 37B 2019-12-09T09:43:21.825 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.YOxd9GUJn2 2019-12-09T09:43:21.876 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:43:21.968 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.WfjC0yw2Yo 2019-12-09T09:43:21.969 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.WfjC0yw2Yo 2019-12-09T09:43:22.039 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.WfjC0yw2Yo 2019-12-09T09:43:22.147 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.WfjC0yw2Yo is 37B 2019-12-09T09:43:22.200 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.WfjC0yw2Yo 2019-12-09T09:43:22.247 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:43:22.339 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.xbF4JodUZG 2019-12-09T09:43:22.339 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.xbF4JodUZG 2019-12-09T09:43:22.415 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.xbF4JodUZG 2019-12-09T09:43:22.541 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.xbF4JodUZG is 37B 2019-12-09T09:43:22.590 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.xbF4JodUZG 2019-12-09T09:43:22.632 INFO:teuthology.orchestra.run.smithi161:> mktemp 2019-12-09T09:43:22.715 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.2nczmYDFX1 2019-12-09T09:43:22.715 INFO:teuthology.orchestra.run.smithi161:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.2nczmYDFX1 2019-12-09T09:43:22.778 INFO:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.2nczmYDFX1 2019-12-09T09:43:22.880 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.2nczmYDFX1 is 37B 2019-12-09T09:43:22.929 INFO:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.2nczmYDFX1 2019-12-09T09:43:22.975 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new ec162fe1-4d44-4b53-a5f3-0129aa5ae42f 0 2019-12-09T09:43:24.230 INFO:teuthology.orchestra.run.smithi161.stdout:0 2019-12-09T09:43:24.257 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 660e1839-d10b-41c3-892f-8358c7d89291 1 2019-12-09T09:43:25.624 INFO:teuthology.orchestra.run.smithi161.stdout:1 2019-12-09T09:43:25.644 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new f5126bf7-29bb-4e26-adc6-6940af3921e0 2 2019-12-09T09:43:26.784 INFO:teuthology.orchestra.run.smithi161.stdout:2 2019-12-09T09:43:26.811 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 60bb73b5-035e-4f5a-b59e-6a3d2e972d51 3 2019-12-09T09:43:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:3 2019-12-09T09:43:27.935 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2 4 2019-12-09T09:43:29.014 INFO:teuthology.orchestra.run.smithi161.stdout:4 2019-12-09T09:43:29.040 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 65015e02-bc89-4179-b216-813159bcdd3a 5 2019-12-09T09:43:29.957 INFO:teuthology.orchestra.run.smithi161.stdout:5 2019-12-09T09:43:29.980 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new f9733cdb-3fe0-4fda-b75c-bca3d02eb152 6 2019-12-09T09:43:31.171 INFO:teuthology.orchestra.run.smithi161.stdout:6 2019-12-09T09:43:31.194 INFO:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new b4cd9530-9599-48d7-8a9e-926d87dc574f 7 2019-12-09T09:43:32.332 INFO:teuthology.orchestra.run.smithi161.stdout:7 2019-12-09T09:43:32.359 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.359 INFO:tasks.ceph.osd.0:Restarting daemon 2019-12-09T09:43:32.360 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-12-09T09:43:32.365 INFO:tasks.ceph.osd.0:Started 2019-12-09T09:43:32.365 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.365 INFO:tasks.ceph.osd.1:Restarting daemon 2019-12-09T09:43:32.366 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-12-09T09:43:32.369 INFO:tasks.ceph.osd.1:Started 2019-12-09T09:43:32.369 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.369 INFO:tasks.ceph.osd.2:Restarting daemon 2019-12-09T09:43:32.370 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-12-09T09:43:32.373 INFO:tasks.ceph.osd.2:Started 2019-12-09T09:43:32.373 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.373 INFO:tasks.ceph.osd.3:Restarting daemon 2019-12-09T09:43:32.374 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-12-09T09:43:32.376 INFO:tasks.ceph.osd.3:Started 2019-12-09T09:43:32.376 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.377 INFO:tasks.ceph.osd.4:Restarting daemon 2019-12-09T09:43:32.377 INFO:teuthology.orchestra.run.smithi161:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-12-09T09:43:32.381 INFO:tasks.ceph.osd.4:Started 2019-12-09T09:43:32.381 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.381 INFO:tasks.ceph.osd.5:Restarting daemon 2019-12-09T09:43:32.382 INFO:teuthology.orchestra.run.smithi161:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-12-09T09:43:32.386 INFO:tasks.ceph.osd.5:Started 2019-12-09T09:43:32.386 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.387 INFO:tasks.ceph.osd.6:Restarting daemon 2019-12-09T09:43:32.387 INFO:teuthology.orchestra.run.smithi161:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-12-09T09:43:32.390 INFO:tasks.ceph.osd.6:Started 2019-12-09T09:43:32.391 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:43:32.391 INFO:tasks.ceph.osd.7:Restarting daemon 2019-12-09T09:43:32.391 INFO:teuthology.orchestra.run.smithi161:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-12-09T09:43:32.395 INFO:tasks.ceph.osd.7:Started 2019-12-09T09:43:32.395 INFO:tasks.ceph:Waiting for OSDs to come up 2019-12-09T09:43:33.202 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:43:34.306 INFO:teuthology.misc.health.smithi059.stdout: 2019-12-09T09:43:34.306 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":10,"fsid":"9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf","created":"2019-12-09T09:43:16.982803+0000","modified":"2019-12-09T09:43:32.147155+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"ec162fe1-4d44-4b53-a5f3-0129aa5ae42f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"660e1839-d10b-41c3-892f-8358c7d89291","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f5126bf7-29bb-4e26-adc6-6940af3921e0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60bb73b5-035e-4f5a-b59e-6a3d2e972d51","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"65015e02-bc89-4179-b216-813159bcdd3a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f9733cdb-3fe0-4fda-b75c-bca3d02eb152","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b4cd9530-9599-48d7-8a9e-926d87dc574f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:43:34.320 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-09T09:43:36.180 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:43:36.170+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.216 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T09:43:36.200+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.224 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T09:43:36.215+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.232 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T09:43:36.219+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.408 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T09:43:36.397+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.417 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T09:43:36.407+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.507 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T09:43:36.496+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:36.530 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T09:43:36.517+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:43:37.124 INFO:tasks.ceph.osd.7.smithi161.stderr:==00:00:00:04.596 9943== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.190 INFO:tasks.ceph.osd.6.smithi161.stderr:==00:00:00:04.665 9942== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.223 INFO:tasks.ceph.osd.5.smithi161.stderr:==00:00:00:04.711 9941== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.245 INFO:tasks.ceph.osd.4.smithi161.stderr:==00:00:00:04.730 9940== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.409 INFO:tasks.ceph.osd.1.smithi059.stderr:==00:00:00:04.917 9847== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.420 INFO:tasks.ceph.osd.0.smithi059.stderr:==00:00:00:04.959 9835== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.518 INFO:tasks.ceph.osd.2.smithi059.stderr:==00:00:00:05.024 9848== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.554 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T09:43:37.535+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.582 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T09:43:37.566+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.582 INFO:tasks.ceph.osd.3.smithi059.stderr:==00:00:00:05.074 9849== Warning: unimplemented fcntl command: 1036 2019-12-09T09:43:37.588 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T09:43:37.554+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.607 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T09:43:37.576+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.616 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T09:43:37.601+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.623 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T09:43:37.620+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.624 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T09:43:37.579+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.637 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T09:43:37.574+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.643 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T09:43:37.625+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.651 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:43:37.639+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.660 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:43:37.654+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.685 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:43:37.658+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.784 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T09:43:37.771+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.796 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T09:43:37.775+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.804 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T09:43:37.787+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.810 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T09:43:37.790+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.835 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T09:43:37.791+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.839 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T09:43:37.796+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.951 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T09:43:37.922+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:37.965 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T09:43:37.938+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:37.984 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T09:43:37.941+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:38.050 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T09:43:38.043+0000 9b13800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-12-09T09:43:38.082 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T09:43:38.064+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:38.119 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T09:43:38.068+0000 9b13800 -1 journal do_read_entry(8192): bad header magic 2019-12-09T09:43:38.180 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T09:43:38.179+0000 9b13800 -1 osd.6 0 log_to_monitors {default=true} 2019-12-09T09:43:38.233 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T09:43:38.231+0000 9b13800 -1 osd.7 0 log_to_monitors {default=true} 2019-12-09T09:43:38.236 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T09:43:38.234+0000 9b13800 -1 osd.4 0 log_to_monitors {default=true} 2019-12-09T09:43:38.279 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:43:38.278+0000 9b13800 -1 osd.5 0 log_to_monitors {default=true} 2019-12-09T09:43:38.400 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T09:43:38.399+0000 9b13800 -1 osd.1 0 log_to_monitors {default=true} 2019-12-09T09:43:38.518 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T09:43:38.515+0000 9b13800 -1 osd.0 0 log_to_monitors {default=true} 2019-12-09T09:43:38.646 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T09:43:38.644+0000 9b13800 -1 osd.2 0 log_to_monitors {default=true} 2019-12-09T09:43:38.711 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T09:43:38.709+0000 9b13800 -1 osd.3 0 log_to_monitors {default=true} 2019-12-09T09:43:39.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T09:43:39.929+0000 29a5f700 -1 osd.4 0 waiting for initial osdmap 2019-12-09T09:43:39.937 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T09:43:39.935+0000 29a5f700 -1 osd.6 0 waiting for initial osdmap 2019-12-09T09:43:41.010 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:43:41.006+0000 29a5f700 -1 osd.5 0 waiting for initial osdmap 2019-12-09T09:43:41.023 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T09:43:41.017+0000 29a5f700 -1 osd.1 0 waiting for initial osdmap 2019-12-09T09:43:41.044 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T09:43:41.039+0000 29a5f700 -1 osd.0 0 waiting for initial osdmap 2019-12-09T09:43:41.083 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T09:43:41.077+0000 29a5f700 -1 osd.2 0 waiting for initial osdmap 2019-12-09T09:43:41.131 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:43:41.139 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T09:43:41.129+0000 29a5f700 -1 osd.3 0 waiting for initial osdmap 2019-12-09T09:43:41.230 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T09:43:41.216+0000 29a5f700 -1 osd.7 0 waiting for initial osdmap 2019-12-09T09:43:41.927 INFO:teuthology.misc.health.smithi059.stdout: 2019-12-09T09:43:41.927 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":13,"fsid":"9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf","created":"2019-12-09T09:43:16.982803+0000","modified":"2019-12-09T09:43:40.523081+0000","last_up_change":"2019-12-09T09:43:40.523081+0000","last_in_change":"2019-12-09T09:43:40.523081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"ec162fe1-4d44-4b53-a5f3-0129aa5ae42f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"660e1839-d10b-41c3-892f-8358c7d89291","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f5126bf7-29bb-4e26-adc6-6940af3921e0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60bb73b5-035e-4f5a-b59e-6a3d2e972d51","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":9940},{"type":"v1","addr":"172.21.15.161:6809","nonce":9940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":9940},{"type":"v1","addr":"172.21.15.161:6812","nonce":9940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":9940},{"type":"v1","addr":"172.21.15.161:6822","nonce":9940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":9940},{"type":"v1","addr":"172.21.15.161:6817","nonce":9940}]},"public_addr":"172.21.15.161:6809/9940","cluster_addr":"172.21.15.161:6812/9940","heartbeat_back_addr":"172.21.15.161:6822/9940","heartbeat_front_addr":"172.21.15.161:6817/9940","state":["exists","up"]},{"osd":5,"uuid":"65015e02-bc89-4179-b216-813159bcdd3a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f9733cdb-3fe0-4fda-b75c-bca3d02eb152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":9942},{"type":"v1","addr":"172.21.15.161:6813","nonce":9942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":9942},{"type":"v1","addr":"172.21.15.161:6819","nonce":9942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6827","nonce":9942},{"type":"v1","addr":"172.21.15.161:6829","nonce":9942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":9942},{"type":"v1","addr":"172.21.15.161:6825","nonce":9942}]},"public_addr":"172.21.15.161:6813/9942","cluster_addr":"172.21.15.161:6819/9942","heartbeat_back_addr":"172.21.15.161:6829/9942","heartbeat_front_addr":"172.21.15.161:6825/9942","state":["exists","up"]},{"osd":7,"uuid":"b4cd9530-9599-48d7-8a9e-926d87dc574f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:43:41.951 DEBUG:teuthology.misc:2 of 8 OSDs are up 2019-12-09T09:43:48.763 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:43:49.792 INFO:teuthology.misc.health.smithi059.stdout: 2019-12-09T09:43:49.792 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":15,"fsid":"9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf","created":"2019-12-09T09:43:16.982803+0000","modified":"2019-12-09T09:43:43.079489+0000","last_up_change":"2019-12-09T09:43:41.859223+0000","last_in_change":"2019-12-09T09:43:41.859223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"ec162fe1-4d44-4b53-a5f3-0129aa5ae42f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":9835},{"type":"v1","addr":"172.21.15.59:6819","nonce":9835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":9835},{"type":"v1","addr":"172.21.15.59:6821","nonce":9835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":9835},{"type":"v1","addr":"172.21.15.59:6826","nonce":9835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":9835},{"type":"v1","addr":"172.21.15.59:6823","nonce":9835}]},"public_addr":"172.21.15.59:6819/9835","cluster_addr":"172.21.15.59:6821/9835","heartbeat_back_addr":"172.21.15.59:6826/9835","heartbeat_front_addr":"172.21.15.59:6823/9835","state":["exists","up"]},{"osd":1,"uuid":"660e1839-d10b-41c3-892f-8358c7d89291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":9847},{"type":"v1","addr":"172.21.15.59:6803","nonce":9847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":9847},{"type":"v1","addr":"172.21.15.59:6805","nonce":9847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":9847},{"type":"v1","addr":"172.21.15.59:6811","nonce":9847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":9847},{"type":"v1","addr":"172.21.15.59:6807","nonce":9847}]},"public_addr":"172.21.15.59:6803/9847","cluster_addr":"172.21.15.59:6805/9847","heartbeat_back_addr":"172.21.15.59:6811/9847","heartbeat_front_addr":"172.21.15.59:6807/9847","state":["exists","up"]},{"osd":2,"uuid":"f5126bf7-29bb-4e26-adc6-6940af3921e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":9848},{"type":"v1","addr":"172.21.15.59:6810","nonce":9848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":9848},{"type":"v1","addr":"172.21.15.59:6813","nonce":9848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":9848},{"type":"v1","addr":"172.21.15.59:6817","nonce":9848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":9848},{"type":"v1","addr":"172.21.15.59:6815","nonce":9848}]},"public_addr":"172.21.15.59:6810/9848","cluster_addr":"172.21.15.59:6813/9848","heartbeat_back_addr":"172.21.15.59:6817/9848","heartbeat_front_addr":"172.21.15.59:6815/9848","state":["exists","up"]},{"osd":3,"uuid":"60bb73b5-035e-4f5a-b59e-6a3d2e972d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":9849},{"type":"v1","addr":"172.21.15.59:6827","nonce":9849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":9849},{"type":"v1","addr":"172.21.15.59:6829","nonce":9849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":9849},{"type":"v1","addr":"172.21.15.59:6833","nonce":9849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":9849},{"type":"v1","addr":"172.21.15.59:6831","nonce":9849}]},"public_addr":"172.21.15.59:6827/9849","cluster_addr":"172.21.15.59:6829/9849","heartbeat_back_addr":"172.21.15.59:6833/9849","heartbeat_front_addr":"172.21.15.59:6831/9849","state":["exists","up"]},{"osd":4,"uuid":"e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":9940},{"type":"v1","addr":"172.21.15.161:6809","nonce":9940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":9940},{"type":"v1","addr":"172.21.15.161:6812","nonce":9940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":9940},{"type":"v1","addr":"172.21.15.161:6822","nonce":9940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":9940},{"type":"v1","addr":"172.21.15.161:6817","nonce":9940}]},"public_addr":"172.21.15.161:6809/9940","cluster_addr":"172.21.15.161:6812/9940","heartbeat_back_addr":"172.21.15.161:6822/9940","heartbeat_front_addr":"172.21.15.161:6817/9940","state":["exists","up"]},{"osd":5,"uuid":"65015e02-bc89-4179-b216-813159bcdd3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":9941},{"type":"v1","addr":"172.21.15.161:6801","nonce":9941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":9941},{"type":"v1","addr":"172.21.15.161:6803","nonce":9941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":9941},{"type":"v1","addr":"172.21.15.161:6807","nonce":9941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":9941},{"type":"v1","addr":"172.21.15.161:6805","nonce":9941}]},"public_addr":"172.21.15.161:6801/9941","cluster_addr":"172.21.15.161:6803/9941","heartbeat_back_addr":"172.21.15.161:6807/9941","heartbeat_front_addr":"172.21.15.161:6805/9941","state":["exists","up"]},{"osd":6,"uuid":"f9733cdb-3fe0-4fda-b75c-bca3d02eb152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":9942},{"type":"v1","addr":"172.21.15.161:6813","nonce":9942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":9942},{"type":"v1","addr":"172.21.15.161:6819","nonce":9942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6827","nonce":9942},{"type":"v1","addr":"172.21.15.161:6829","nonce":9942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":9942},{"type":"v1","addr":"172.21.15.161:6825","nonce":9942}]},"public_addr":"172.21.15.161:6813/9942","cluster_addr":"172.21.15.161:6819/9942","heartbeat_back_addr":"172.21.15.161:6829/9942","heartbeat_front_addr":"172.21.15.161:6825/9942","state":["exists","up"]},{"osd":7,"uuid":"b4cd9530-9599-48d7-8a9e-926d87dc574f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":9943},{"type":"v1","addr":"172.21.15.161:6816","nonce":9943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":9943},{"type":"v1","addr":"172.21.15.161:6824","nonce":9943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":9943},{"type":"v1","addr":"172.21.15.161:6831","nonce":9943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":9943},{"type":"v1","addr":"172.21.15.161:6828","nonce":9943}]},"public_addr":"172.21.15.161:6816/9943","cluster_addr":"172.21.15.161:6824/9943","heartbeat_back_addr":"172.21.15.161:6831/9943","heartbeat_front_addr":"172.21.15.161:6828/9943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.654694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.551226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.758569+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.833408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.352296+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.401802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.335310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.341680+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:43:49.805 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-12-09T09:43:49.805 INFO:tasks.ceph:Creating RBD pool 2019-12-09T09:43:49.805 INFO:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd pool create rbd 8 2019-12-09T09:43:51.365 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'rbd' created 2019-12-09T09:43:51.385 INFO:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-12-09T09:43:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:enabled application 'rbd' on pool 'rbd' 2019-12-09T09:43:53.345 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-12-09T09:43:53.345 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:43:53.346 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-12-09T09:43:54.116 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:43:54.116 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":18,"fsid":"9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf","created":"2019-12-09T09:43:16.982803+0000","modified":"2019-12-09T09:43:53.114607+0000","last_up_change":"2019-12-09T09:43:41.859223+0000","last_in_change":"2019-12-09T09:43:41.859223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:43:50.558162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ec162fe1-4d44-4b53-a5f3-0129aa5ae42f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":9835},{"type":"v1","addr":"172.21.15.59:6819","nonce":9835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":9835},{"type":"v1","addr":"172.21.15.59:6821","nonce":9835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":9835},{"type":"v1","addr":"172.21.15.59:6826","nonce":9835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":9835},{"type":"v1","addr":"172.21.15.59:6823","nonce":9835}]},"public_addr":"172.21.15.59:6819/9835","cluster_addr":"172.21.15.59:6821/9835","heartbeat_back_addr":"172.21.15.59:6826/9835","heartbeat_front_addr":"172.21.15.59:6823/9835","state":["exists","up"]},{"osd":1,"uuid":"660e1839-d10b-41c3-892f-8358c7d89291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":9847},{"type":"v1","addr":"172.21.15.59:6803","nonce":9847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":9847},{"type":"v1","addr":"172.21.15.59:6805","nonce":9847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":9847},{"type":"v1","addr":"172.21.15.59:6811","nonce":9847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":9847},{"type":"v1","addr":"172.21.15.59:6807","nonce":9847}]},"public_addr":"172.21.15.59:6803/9847","cluster_addr":"172.21.15.59:6805/9847","heartbeat_back_addr":"172.21.15.59:6811/9847","heartbeat_front_addr":"172.21.15.59:6807/9847","state":["exists","up"]},{"osd":2,"uuid":"f5126bf7-29bb-4e26-adc6-6940af3921e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":9848},{"type":"v1","addr":"172.21.15.59:6810","nonce":9848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":9848},{"type":"v1","addr":"172.21.15.59:6813","nonce":9848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":9848},{"type":"v1","addr":"172.21.15.59:6817","nonce":9848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":9848},{"type":"v1","addr":"172.21.15.59:6815","nonce":9848}]},"public_addr":"172.21.15.59:6810/9848","cluster_addr":"172.21.15.59:6813/9848","heartbeat_back_addr":"172.21.15.59:6817/9848","heartbeat_front_addr":"172.21.15.59:6815/9848","state":["exists","up"]},{"osd":3,"uuid":"60bb73b5-035e-4f5a-b59e-6a3d2e972d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":9849},{"type":"v1","addr":"172.21.15.59:6827","nonce":9849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":9849},{"type":"v1","addr":"172.21.15.59:6829","nonce":9849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":9849},{"type":"v1","addr":"172.21.15.59:6833","nonce":9849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":9849},{"type":"v1","addr":"172.21.15.59:6831","nonce":9849}]},"public_addr":"172.21.15.59:6827/9849","cluster_addr":"172.21.15.59:6829/9849","heartbeat_back_addr":"172.21.15.59:6833/9849","heartbeat_front_addr":"172.21.15.59:6831/9849","state":["exists","up"]},{"osd":4,"uuid":"e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":9940},{"type":"v1","addr":"172.21.15.161:6809","nonce":9940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":9940},{"type":"v1","addr":"172.21.15.161:6812","nonce":9940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":9940},{"type":"v1","addr":"172.21.15.161:6822","nonce":9940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":9940},{"type":"v1","addr":"172.21.15.161:6817","nonce":9940}]},"public_addr":"172.21.15.161:6809/9940","cluster_addr":"172.21.15.161:6812/9940","heartbeat_back_addr":"172.21.15.161:6822/9940","heartbeat_front_addr":"172.21.15.161:6817/9940","state":["exists","up"]},{"osd":5,"uuid":"65015e02-bc89-4179-b216-813159bcdd3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":9941},{"type":"v1","addr":"172.21.15.161:6801","nonce":9941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":9941},{"type":"v1","addr":"172.21.15.161:6803","nonce":9941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":9941},{"type":"v1","addr":"172.21.15.161:6807","nonce":9941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":9941},{"type":"v1","addr":"172.21.15.161:6805","nonce":9941}]},"public_addr":"172.21.15.161:6801/9941","cluster_addr":"172.21.15.161:6803/9941","heartbeat_back_addr":"172.21.15.161:6807/9941","heartbeat_front_addr":"172.21.15.161:6805/9941","state":["exists","up"]},{"osd":6,"uuid":"f9733cdb-3fe0-4fda-b75c-bca3d02eb152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":9942},{"type":"v1","addr":"172.21.15.161:6813","nonce":9942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":9942},{"type":"v1","addr":"172.21.15.161:6819","nonce":9942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6827","nonce":9942},{"type":"v1","addr":"172.21.15.161:6829","nonce":9942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":9942},{"type":"v1","addr":"172.21.15.161:6825","nonce":9942}]},"public_addr":"172.21.15.161:6813/9942","cluster_addr":"172.21.15.161:6819/9942","heartbeat_back_addr":"172.21.15.161:6829/9942","heartbeat_front_addr":"172.21.15.161:6825/9942","state":["exists","up"]},{"osd":7,"uuid":"b4cd9530-9599-48d7-8a9e-926d87dc574f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":9943},{"type":"v1","addr":"172.21.15.161:6816","nonce":9943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":9943},{"type":"v1","addr":"172.21.15.161:6824","nonce":9943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":9943},{"type":"v1","addr":"172.21.15.161:6831","nonce":9943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":9943},{"type":"v1","addr":"172.21.15.161:6828","nonce":9943}]},"public_addr":"172.21.15.161:6816/9943","cluster_addr":"172.21.15.161:6824/9943","heartbeat_back_addr":"172.21.15.161:6831/9943","heartbeat_front_addr":"172.21.15.161:6828/9943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.654694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.551226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.758569+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.833408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.352296+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.401802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.335310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.341680+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:43:54.141 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-12-09T09:43:50.558162+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'18', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-12-09T09:43:54.142 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-12-09T09:43:54.965 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 8 2019-12-09T09:43:54.987 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-12-09T09:43:54.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-12-09T09:43:54.987 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-12-09T09:43:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:43:56.214 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":8893},{"type":"v1","addr":"172.21.15.59:6801","nonce":8893}]},"active_addr":"172.21.15.59:6801/8893","active_change":"2019-12-09T09:43:21.113344+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-12-09T09:43:56.219 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-12-09T09:43:56.220 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-12-09T09:43:56.220 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:43:57.176 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:43:57.176 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf","created":"2019-12-09T09:43:16.982803+0000","modified":"2019-12-09T09:43:54.198968+0000","last_up_change":"2019-12-09T09:43:41.859223+0000","last_in_change":"2019-12-09T09:43:41.859223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:43:50.558162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ec162fe1-4d44-4b53-a5f3-0129aa5ae42f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":9835},{"type":"v1","addr":"172.21.15.59:6819","nonce":9835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":9835},{"type":"v1","addr":"172.21.15.59:6821","nonce":9835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":9835},{"type":"v1","addr":"172.21.15.59:6826","nonce":9835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":9835},{"type":"v1","addr":"172.21.15.59:6823","nonce":9835}]},"public_addr":"172.21.15.59:6819/9835","cluster_addr":"172.21.15.59:6821/9835","heartbeat_back_addr":"172.21.15.59:6826/9835","heartbeat_front_addr":"172.21.15.59:6823/9835","state":["exists","up"]},{"osd":1,"uuid":"660e1839-d10b-41c3-892f-8358c7d89291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":9847},{"type":"v1","addr":"172.21.15.59:6803","nonce":9847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":9847},{"type":"v1","addr":"172.21.15.59:6805","nonce":9847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":9847},{"type":"v1","addr":"172.21.15.59:6811","nonce":9847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":9847},{"type":"v1","addr":"172.21.15.59:6807","nonce":9847}]},"public_addr":"172.21.15.59:6803/9847","cluster_addr":"172.21.15.59:6805/9847","heartbeat_back_addr":"172.21.15.59:6811/9847","heartbeat_front_addr":"172.21.15.59:6807/9847","state":["exists","up"]},{"osd":2,"uuid":"f5126bf7-29bb-4e26-adc6-6940af3921e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":9848},{"type":"v1","addr":"172.21.15.59:6810","nonce":9848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":9848},{"type":"v1","addr":"172.21.15.59:6813","nonce":9848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":9848},{"type":"v1","addr":"172.21.15.59:6817","nonce":9848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":9848},{"type":"v1","addr":"172.21.15.59:6815","nonce":9848}]},"public_addr":"172.21.15.59:6810/9848","cluster_addr":"172.21.15.59:6813/9848","heartbeat_back_addr":"172.21.15.59:6817/9848","heartbeat_front_addr":"172.21.15.59:6815/9848","state":["exists","up"]},{"osd":3,"uuid":"60bb73b5-035e-4f5a-b59e-6a3d2e972d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":9849},{"type":"v1","addr":"172.21.15.59:6827","nonce":9849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":9849},{"type":"v1","addr":"172.21.15.59:6829","nonce":9849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":9849},{"type":"v1","addr":"172.21.15.59:6833","nonce":9849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":9849},{"type":"v1","addr":"172.21.15.59:6831","nonce":9849}]},"public_addr":"172.21.15.59:6827/9849","cluster_addr":"172.21.15.59:6829/9849","heartbeat_back_addr":"172.21.15.59:6833/9849","heartbeat_front_addr":"172.21.15.59:6831/9849","state":["exists","up"]},{"osd":4,"uuid":"e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":9940},{"type":"v1","addr":"172.21.15.161:6809","nonce":9940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":9940},{"type":"v1","addr":"172.21.15.161:6812","nonce":9940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":9940},{"type":"v1","addr":"172.21.15.161:6822","nonce":9940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":9940},{"type":"v1","addr":"172.21.15.161:6817","nonce":9940}]},"public_addr":"172.21.15.161:6809/9940","cluster_addr":"172.21.15.161:6812/9940","heartbeat_back_addr":"172.21.15.161:6822/9940","heartbeat_front_addr":"172.21.15.161:6817/9940","state":["exists","up"]},{"osd":5,"uuid":"65015e02-bc89-4179-b216-813159bcdd3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":9941},{"type":"v1","addr":"172.21.15.161:6801","nonce":9941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":9941},{"type":"v1","addr":"172.21.15.161:6803","nonce":9941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":9941},{"type":"v1","addr":"172.21.15.161:6807","nonce":9941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":9941},{"type":"v1","addr":"172.21.15.161:6805","nonce":9941}]},"public_addr":"172.21.15.161:6801/9941","cluster_addr":"172.21.15.161:6803/9941","heartbeat_back_addr":"172.21.15.161:6807/9941","heartbeat_front_addr":"172.21.15.161:6805/9941","state":["exists","up"]},{"osd":6,"uuid":"f9733cdb-3fe0-4fda-b75c-bca3d02eb152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":9942},{"type":"v1","addr":"172.21.15.161:6813","nonce":9942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":9942},{"type":"v1","addr":"172.21.15.161:6819","nonce":9942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6827","nonce":9942},{"type":"v1","addr":"172.21.15.161:6829","nonce":9942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":9942},{"type":"v1","addr":"172.21.15.161:6825","nonce":9942}]},"public_addr":"172.21.15.161:6813/9942","cluster_addr":"172.21.15.161:6819/9942","heartbeat_back_addr":"172.21.15.161:6829/9942","heartbeat_front_addr":"172.21.15.161:6825/9942","state":["exists","up"]},{"osd":7,"uuid":"b4cd9530-9599-48d7-8a9e-926d87dc574f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":9943},{"type":"v1","addr":"172.21.15.161:6816","nonce":9943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":9943},{"type":"v1","addr":"172.21.15.161:6824","nonce":9943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":9943},{"type":"v1","addr":"172.21.15.161:6831","nonce":9943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":9943},{"type":"v1","addr":"172.21.15.161:6828","nonce":9943}]},"public_addr":"172.21.15.161:6816/9943","cluster_addr":"172.21.15.161:6824/9943","heartbeat_back_addr":"172.21.15.161:6831/9943","heartbeat_front_addr":"172.21.15.161:6828/9943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.654694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.551226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.758569+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.833408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.352296+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.401802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.335310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.341680+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:43:57.202 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-12-09T09:43:57.203 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:43:58.074 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:43:58.121 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"9b1c3d68-77c7-4bf6-a5cd-b4287f4ef7cf","created":"2019-12-09T09:43:16.982803+0000","modified":"2019-12-09T09:43:54.198968+0000","last_up_change":"2019-12-09T09:43:41.859223+0000","last_in_change":"2019-12-09T09:43:41.859223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:43:50.558162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ec162fe1-4d44-4b53-a5f3-0129aa5ae42f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":9835},{"type":"v1","addr":"172.21.15.59:6819","nonce":9835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":9835},{"type":"v1","addr":"172.21.15.59:6821","nonce":9835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":9835},{"type":"v1","addr":"172.21.15.59:6826","nonce":9835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":9835},{"type":"v1","addr":"172.21.15.59:6823","nonce":9835}]},"public_addr":"172.21.15.59:6819/9835","cluster_addr":"172.21.15.59:6821/9835","heartbeat_back_addr":"172.21.15.59:6826/9835","heartbeat_front_addr":"172.21.15.59:6823/9835","state":["exists","up"]},{"osd":1,"uuid":"660e1839-d10b-41c3-892f-8358c7d89291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":9847},{"type":"v1","addr":"172.21.15.59:6803","nonce":9847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":9847},{"type":"v1","addr":"172.21.15.59:6805","nonce":9847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":9847},{"type":"v1","addr":"172.21.15.59:6811","nonce":9847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":9847},{"type":"v1","addr":"172.21.15.59:6807","nonce":9847}]},"public_addr":"172.21.15.59:6803/9847","cluster_addr":"172.21.15.59:6805/9847","heartbeat_back_addr":"172.21.15.59:6811/9847","heartbeat_front_addr":"172.21.15.59:6807/9847","state":["exists","up"]},{"osd":2,"uuid":"f5126bf7-29bb-4e26-adc6-6940af3921e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":9848},{"type":"v1","addr":"172.21.15.59:6810","nonce":9848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":9848},{"type":"v1","addr":"172.21.15.59:6813","nonce":9848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":9848},{"type":"v1","addr":"172.21.15.59:6817","nonce":9848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":9848},{"type":"v1","addr":"172.21.15.59:6815","nonce":9848}]},"public_addr":"172.21.15.59:6810/9848","cluster_addr":"172.21.15.59:6813/9848","heartbeat_back_addr":"172.21.15.59:6817/9848","heartbeat_front_addr":"172.21.15.59:6815/9848","state":["exists","up"]},{"osd":3,"uuid":"60bb73b5-035e-4f5a-b59e-6a3d2e972d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":9849},{"type":"v1","addr":"172.21.15.59:6827","nonce":9849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":9849},{"type":"v1","addr":"172.21.15.59:6829","nonce":9849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":9849},{"type":"v1","addr":"172.21.15.59:6833","nonce":9849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":9849},{"type":"v1","addr":"172.21.15.59:6831","nonce":9849}]},"public_addr":"172.21.15.59:6827/9849","cluster_addr":"172.21.15.59:6829/9849","heartbeat_back_addr":"172.21.15.59:6833/9849","heartbeat_front_addr":"172.21.15.59:6831/9849","state":["exists","up"]},{"osd":4,"uuid":"e6c3bb4e-9c4d-47bf-ba9e-14fc489444d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":9940},{"type":"v1","addr":"172.21.15.161:6809","nonce":9940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":9940},{"type":"v1","addr":"172.21.15.161:6812","nonce":9940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":9940},{"type":"v1","addr":"172.21.15.161:6822","nonce":9940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":9940},{"type":"v1","addr":"172.21.15.161:6817","nonce":9940}]},"public_addr":"172.21.15.161:6809/9940","cluster_addr":"172.21.15.161:6812/9940","heartbeat_back_addr":"172.21.15.161:6822/9940","heartbeat_front_addr":"172.21.15.161:6817/9940","state":["exists","up"]},{"osd":5,"uuid":"65015e02-bc89-4179-b216-813159bcdd3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":9941},{"type":"v1","addr":"172.21.15.161:6801","nonce":9941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":9941},{"type":"v1","addr":"172.21.15.161:6803","nonce":9941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":9941},{"type":"v1","addr":"172.21.15.161:6807","nonce":9941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":9941},{"type":"v1","addr":"172.21.15.161:6805","nonce":9941}]},"public_addr":"172.21.15.161:6801/9941","cluster_addr":"172.21.15.161:6803/9941","heartbeat_back_addr":"172.21.15.161:6807/9941","heartbeat_front_addr":"172.21.15.161:6805/9941","state":["exists","up"]},{"osd":6,"uuid":"f9733cdb-3fe0-4fda-b75c-bca3d02eb152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":9942},{"type":"v1","addr":"172.21.15.161:6813","nonce":9942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":9942},{"type":"v1","addr":"172.21.15.161:6819","nonce":9942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6827","nonce":9942},{"type":"v1","addr":"172.21.15.161:6829","nonce":9942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":9942},{"type":"v1","addr":"172.21.15.161:6825","nonce":9942}]},"public_addr":"172.21.15.161:6813/9942","cluster_addr":"172.21.15.161:6819/9942","heartbeat_back_addr":"172.21.15.161:6829/9942","heartbeat_front_addr":"172.21.15.161:6825/9942","state":["exists","up"]},{"osd":7,"uuid":"b4cd9530-9599-48d7-8a9e-926d87dc574f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":9943},{"type":"v1","addr":"172.21.15.161:6816","nonce":9943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":9943},{"type":"v1","addr":"172.21.15.161:6824","nonce":9943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":9943},{"type":"v1","addr":"172.21.15.161:6831","nonce":9943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":9943},{"type":"v1","addr":"172.21.15.161:6828","nonce":9943}]},"public_addr":"172.21.15.161:6816/9943","cluster_addr":"172.21.15.161:6824/9943","heartbeat_back_addr":"172.21.15.161:6831/9943","heartbeat_front_addr":"172.21.15.161:6828/9943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.654694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.551226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.758569+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.833408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.352296+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.401802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.335310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:43:39.341680+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:43:58.123 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-12-09T09:43:58.737 INFO:teuthology.orchestra.run.smithi059.stdout:60129542150 2019-12-09T09:43:58.737 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-12-09T09:43:59.360 INFO:teuthology.orchestra.run.smithi059.stdout:60129542149 2019-12-09T09:43:59.361 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-12-09T09:43:59.919 INFO:teuthology.orchestra.run.smithi059.stdout:60129542150 2019-12-09T09:43:59.920 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-12-09T09:44:00.418 INFO:teuthology.orchestra.run.smithi059.stdout:60129542150 2019-12-09T09:44:00.419 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-12-09T09:44:01.073 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2019-12-09T09:44:01.074 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-12-09T09:44:01.767 INFO:teuthology.orchestra.run.smithi059.stdout:60129542149 2019-12-09T09:44:01.767 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-12-09T09:44:02.348 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2019-12-09T09:44:02.349 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-12-09T09:44:03.029 INFO:teuthology.orchestra.run.smithi059.stdout:60129542150 2019-12-09T09:44:03.030 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-12-09T09:44:03.990 INFO:teuthology.orchestra.run.smithi059.stdout:60129542151 2019-12-09T09:44:04.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.0 2019-12-09T09:44:04.008 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-12-09T09:44:04.891 INFO:teuthology.orchestra.run.smithi059.stdout:60129542150 2019-12-09T09:44:04.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.1 2019-12-09T09:44:04.905 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-12-09T09:44:05.641 INFO:teuthology.orchestra.run.smithi059.stdout:60129542151 2019-12-09T09:44:05.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.2 2019-12-09T09:44:05.660 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-12-09T09:44:06.597 INFO:teuthology.orchestra.run.smithi059.stdout:60129542151 2019-12-09T09:44:06.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.3 2019-12-09T09:44:06.620 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-12-09T09:44:07.488 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2019-12-09T09:44:07.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.4 2019-12-09T09:44:07.506 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-12-09T09:44:08.323 INFO:teuthology.orchestra.run.smithi059.stdout:60129542150 2019-12-09T09:44:08.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.5 2019-12-09T09:44:08.361 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-12-09T09:44:09.114 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2019-12-09T09:44:09.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.6 2019-12-09T09:44:09.131 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-12-09T09:44:10.098 INFO:teuthology.orchestra.run.smithi059.stdout:60129542152 2019-12-09T09:44:10.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542152 for osd.7 2019-12-09T09:44:10.113 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-12-09T09:44:10.113 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T09:44:11.021 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2019-12-09T09:44:11.021 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:44:11.035 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-12-09T09:44:09.365768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699584,"kb_used_data":747895456,"kb_used_omap":294,"kb_used_meta":0,"kb_avail":747895456,"statfs":{"total":767585320960,"available":765844946944,"internally_reserved":0,"allocated":765844946944,"data_stored":765844946944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":301745,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.690580+0000","last_change":"2019-12-09T09:43:52.851217+0000","last_active":"2019-12-09T09:43:53.690580+0000","last_peered":"2019-12-09T09:43:53.690580+0000","last_clean":"2019-12-09T09:43:53.690580+0000","last_became_active":"2019-12-09T09:43:52.832133+0000","last_became_peered":"2019-12-09T09:43:52.832133+0000","last_unstale":"2019-12-09T09:43:53.690580+0000","last_undegraded":"2019-12-09T09:43:53.690580+0000","last_fullsized":"2019-12-09T09:43:53.690580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.708086+0000","last_change":"2019-12-09T09:43:53.698752+0000","last_active":"2019-12-09T09:43:53.708086+0000","last_peered":"2019-12-09T09:43:53.708086+0000","last_clean":"2019-12-09T09:43:53.708086+0000","last_became_active":"2019-12-09T09:43:53.669760+0000","last_became_peered":"2019-12-09T09:43:53.669760+0000","last_unstale":"2019-12-09T09:43:53.708086+0000","last_undegraded":"2019-12-09T09:43:53.708086+0000","last_fullsized":"2019-12-09T09:43:53.708086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.650519+0000","last_change":"2019-12-09T09:43:53.623818+0000","last_active":"2019-12-09T09:43:53.650519+0000","last_peered":"2019-12-09T09:43:53.650519+0000","last_clean":"2019-12-09T09:43:53.650519+0000","last_became_active":"2019-12-09T09:43:53.580221+0000","last_became_peered":"2019-12-09T09:43:53.580221+0000","last_unstale":"2019-12-09T09:43:53.650519+0000","last_undegraded":"2019-12-09T09:43:53.650519+0000","last_fullsized":"2019-12-09T09:43:53.650519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:54.014169+0000","last_change":"2019-12-09T09:43:52.498169+0000","last_active":"2019-12-09T09:43:54.014169+0000","last_peered":"2019-12-09T09:43:54.014169+0000","last_clean":"2019-12-09T09:43:54.014169+0000","last_became_active":"2019-12-09T09:43:52.457959+0000","last_became_peered":"2019-12-09T09:43:52.457959+0000","last_unstale":"2019-12-09T09:43:54.014169+0000","last_undegraded":"2019-12-09T09:43:54.014169+0000","last_fullsized":"2019-12-09T09:43:54.014169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.680583+0000","last_change":"2019-12-09T09:43:52.638993+0000","last_active":"2019-12-09T09:43:53.680583+0000","last_peered":"2019-12-09T09:43:53.680583+0000","last_clean":"2019-12-09T09:43:53.680583+0000","last_became_active":"2019-12-09T09:43:52.608626+0000","last_became_peered":"2019-12-09T09:43:52.608626+0000","last_unstale":"2019-12-09T09:43:53.680583+0000","last_undegraded":"2019-12-09T09:43:53.680583+0000","last_fullsized":"2019-12-09T09:43:53.680583+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.690269+0000","last_change":"2019-12-09T09:43:53.642946+0000","last_active":"2019-12-09T09:43:53.690269+0000","last_peered":"2019-12-09T09:43:53.690269+0000","last_clean":"2019-12-09T09:43:53.690269+0000","last_became_active":"2019-12-09T09:43:53.614010+0000","last_became_peered":"2019-12-09T09:43:53.614010+0000","last_unstale":"2019-12-09T09:43:53.690269+0000","last_undegraded":"2019-12-09T09:43:53.690269+0000","last_fullsized":"2019-12-09T09:43:53.690269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.971521+0000","last_change":"2019-12-09T09:43:52.553569+0000","last_active":"2019-12-09T09:43:53.971521+0000","last_peered":"2019-12-09T09:43:53.971521+0000","last_clean":"2019-12-09T09:43:53.971521+0000","last_became_active":"2019-12-09T09:43:52.519073+0000","last_became_peered":"2019-12-09T09:43:52.519073+0000","last_unstale":"2019-12-09T09:43:53.971521+0000","last_undegraded":"2019-12-09T09:43:53.971521+0000","last_fullsized":"2019-12-09T09:43:53.971521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.393017+0000","last_change":"2019-12-09T09:43:52.485754+0000","last_active":"2019-12-09T09:43:53.393017+0000","last_peered":"2019-12-09T09:43:53.393017+0000","last_clean":"2019-12-09T09:43:53.393017+0000","last_became_active":"2019-12-09T09:43:52.440100+0000","last_became_peered":"2019-12-09T09:43:52.440100+0000","last_unstale":"2019-12-09T09:43:53.393017+0000","last_undegraded":"2019-12-09T09:43:53.393017+0000","last_fullsized":"2019-12-09T09:43:53.393017+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32897,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.155}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.800}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.334}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.541}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.216}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.362}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.753}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32853,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.862}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.938}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.431}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.132}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":45.400}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37755,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.155}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":8.776}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.736}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.687}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.676}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.894}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.914}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32878,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.114}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.156}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.290}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.112}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.977}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.613}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.799}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37749,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.410}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.942}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.985}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.422}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.818}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.742}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47333,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.743}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.105}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.888}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.149}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.562}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.007}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.299}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42569,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.667}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.802}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.203}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.393}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.182}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.849}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37711,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.098}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.369}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.583}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.308}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.727}]}]}]}} 2019-12-09T09:44:11.037 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T09:44:11.888 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2019-12-09T09:44:11.888 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:44:11.904 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2019-12-09T09:44:11.366441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699584,"kb_used_data":747895456,"kb_used_omap":294,"kb_used_meta":0,"kb_avail":747895456,"statfs":{"total":767585320960,"available":765844946944,"internally_reserved":0,"allocated":765844946944,"data_stored":765844946944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":301745,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.690580+0000","last_change":"2019-12-09T09:43:52.851217+0000","last_active":"2019-12-09T09:43:53.690580+0000","last_peered":"2019-12-09T09:43:53.690580+0000","last_clean":"2019-12-09T09:43:53.690580+0000","last_became_active":"2019-12-09T09:43:52.832133+0000","last_became_peered":"2019-12-09T09:43:52.832133+0000","last_unstale":"2019-12-09T09:43:53.690580+0000","last_undegraded":"2019-12-09T09:43:53.690580+0000","last_fullsized":"2019-12-09T09:43:53.690580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.708086+0000","last_change":"2019-12-09T09:43:53.698752+0000","last_active":"2019-12-09T09:43:53.708086+0000","last_peered":"2019-12-09T09:43:53.708086+0000","last_clean":"2019-12-09T09:43:53.708086+0000","last_became_active":"2019-12-09T09:43:53.669760+0000","last_became_peered":"2019-12-09T09:43:53.669760+0000","last_unstale":"2019-12-09T09:43:53.708086+0000","last_undegraded":"2019-12-09T09:43:53.708086+0000","last_fullsized":"2019-12-09T09:43:53.708086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.650519+0000","last_change":"2019-12-09T09:43:53.623818+0000","last_active":"2019-12-09T09:43:53.650519+0000","last_peered":"2019-12-09T09:43:53.650519+0000","last_clean":"2019-12-09T09:43:53.650519+0000","last_became_active":"2019-12-09T09:43:53.580221+0000","last_became_peered":"2019-12-09T09:43:53.580221+0000","last_unstale":"2019-12-09T09:43:53.650519+0000","last_undegraded":"2019-12-09T09:43:53.650519+0000","last_fullsized":"2019-12-09T09:43:53.650519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:54.014169+0000","last_change":"2019-12-09T09:43:52.498169+0000","last_active":"2019-12-09T09:43:54.014169+0000","last_peered":"2019-12-09T09:43:54.014169+0000","last_clean":"2019-12-09T09:43:54.014169+0000","last_became_active":"2019-12-09T09:43:52.457959+0000","last_became_peered":"2019-12-09T09:43:52.457959+0000","last_unstale":"2019-12-09T09:43:54.014169+0000","last_undegraded":"2019-12-09T09:43:54.014169+0000","last_fullsized":"2019-12-09T09:43:54.014169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.680583+0000","last_change":"2019-12-09T09:43:52.638993+0000","last_active":"2019-12-09T09:43:53.680583+0000","last_peered":"2019-12-09T09:43:53.680583+0000","last_clean":"2019-12-09T09:43:53.680583+0000","last_became_active":"2019-12-09T09:43:52.608626+0000","last_became_peered":"2019-12-09T09:43:52.608626+0000","last_unstale":"2019-12-09T09:43:53.680583+0000","last_undegraded":"2019-12-09T09:43:53.680583+0000","last_fullsized":"2019-12-09T09:43:53.680583+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.690269+0000","last_change":"2019-12-09T09:43:53.642946+0000","last_active":"2019-12-09T09:43:53.690269+0000","last_peered":"2019-12-09T09:43:53.690269+0000","last_clean":"2019-12-09T09:43:53.690269+0000","last_became_active":"2019-12-09T09:43:53.614010+0000","last_became_peered":"2019-12-09T09:43:53.614010+0000","last_unstale":"2019-12-09T09:43:53.690269+0000","last_undegraded":"2019-12-09T09:43:53.690269+0000","last_fullsized":"2019-12-09T09:43:53.690269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.971521+0000","last_change":"2019-12-09T09:43:52.553569+0000","last_active":"2019-12-09T09:43:53.971521+0000","last_peered":"2019-12-09T09:43:53.971521+0000","last_clean":"2019-12-09T09:43:53.971521+0000","last_became_active":"2019-12-09T09:43:52.519073+0000","last_became_peered":"2019-12-09T09:43:52.519073+0000","last_unstale":"2019-12-09T09:43:53.971521+0000","last_undegraded":"2019-12-09T09:43:53.971521+0000","last_fullsized":"2019-12-09T09:43:53.971521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:43:53.393017+0000","last_change":"2019-12-09T09:43:52.485754+0000","last_active":"2019-12-09T09:43:53.393017+0000","last_peered":"2019-12-09T09:43:53.393017+0000","last_clean":"2019-12-09T09:43:53.393017+0000","last_became_active":"2019-12-09T09:43:52.440100+0000","last_became_peered":"2019-12-09T09:43:52.440100+0000","last_unstale":"2019-12-09T09:43:53.393017+0000","last_undegraded":"2019-12-09T09:43:53.393017+0000","last_fullsized":"2019-12-09T09:43:53.393017+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:43:50.787808+0000","last_clean_scrub_stamp":"2019-12-09T09:43:50.787808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32897,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.155}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.800}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.334}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.541}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.216}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.362}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.753}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32853,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.862}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.938}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.431}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.132}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":45.400}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37755,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.155}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":8.776}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.736}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.687}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.676}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.894}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.914}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32878,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.114}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.156}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.290}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.112}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.977}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.613}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.799}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37749,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.410}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.942}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.985}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.422}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.818}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.742}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":46,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47333,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.743}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.105}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.888}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.149}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.562}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.007}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.299}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":41,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42569,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.667}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.802}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.203}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.393}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.182}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.849}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212448,"kb_used_data":93486932,"kb_used_omap":36,"kb_used_meta":0,"kb_avail":93486932,"statfs":{"total":95948165120,"available":95730618368,"internally_reserved":0,"allocated":95730618368,"data_stored":95730618368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37711,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.098}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.369}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.583}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.308}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.727}]}]}]}} 2019-12-09T09:44:11.907 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-12-09T09:44:11.907 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-12-09T09:44:11.908 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-12-09T09:44:11.908 INFO:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-12-09T09:44:12.783 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:44:12.797 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-12-09T09:44:12.797 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-12-09T09:44:12.798 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-12-09T09:44:12.811 DEBUG:teuthology.task:Applying overrides for task openssl_keys: {'rgw.client.1': {'ca': 'root', 'client': 'client.1', 'embed-key': True}, 'root': {'key-type': 'rsa:4096', 'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0', 'client.1']}, 'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}} 2019-12-09T09:44:12.812 INFO:teuthology.orchestra.run.smithi059:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-09T09:44:12.904 INFO:teuthology.orchestra.run.smithi059:> openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/ubuntu/cephtest/ca/root.key -days 365 -out /home/ubuntu/cephtest/ca/root.crt -subj /CN=teuthology 2019-12-09T09:44:12.993 INFO:teuthology.orchestra.run.smithi059.stderr:Generating a 4096 bit RSA private key 2019-12-09T09:44:13.591 INFO:teuthology.orchestra.run.smithi059.stderr:.................................................................................................................................++ 2019-12-09T09:44:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:..................++ 2019-12-09T09:44:13.674 INFO:teuthology.orchestra.run.smithi059.stderr:writing new private key to '/home/ubuntu/cephtest/ca/root.key' 2019-12-09T09:44:13.674 INFO:teuthology.orchestra.run.smithi059.stderr:----- 2019-12-09T09:44:13.684 INFO:teuthology.orchestra.run.smithi059:> sudo cp /home/ubuntu/cephtest/ca/root.crt /usr/share/pki/ca-trust-source/anchors/root.crt 2019-12-09T09:44:13.809 INFO:teuthology.orchestra.run.smithi059:> sudo update-ca-trust 2019-12-09T09:44:14.485 INFO:teuthology.orchestra.run.smithi161:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-09T09:44:14.550 INFO:tasks.openssl_keys:copying from ubuntu@smithi059.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/root.crt to ubuntu@smithi161.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/root.crt... 2019-12-09T09:44:14.631 DEBUG:teuthology.orchestra.remote:smithi059:/home/ubuntu/cephtest/ca/root.crt is 2KB 2019-12-09T09:44:14.810 INFO:teuthology.orchestra.run.smithi161:> sudo mv /home/ubuntu/cephtest/ca/root.crt /usr/share/pki/ca-trust-source/anchors/root.crt 2019-12-09T09:44:14.938 INFO:teuthology.orchestra.run.smithi161:> sudo update-ca-trust 2019-12-09T09:44:15.779 INFO:teuthology.orchestra.run.smithi161:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-09T09:44:15.860 INFO:teuthology.orchestra.run.smithi059:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.1.key -out /home/ubuntu/cephtest/ca/rgw.client.1.csr -subj /CN=smithi161.front.sepia.ceph.com 2019-12-09T09:44:15.958 INFO:teuthology.orchestra.run.smithi059.stderr:Generating a 2048 bit RSA private key 2019-12-09T09:44:16.064 INFO:teuthology.orchestra.run.smithi059.stderr:.............................................+++ 2019-12-09T09:44:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:..................+++ 2019-12-09T09:44:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:writing new private key to '/home/ubuntu/cephtest/ca/rgw.client.1.key' 2019-12-09T09:44:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:----- 2019-12-09T09:44:16.100 INFO:teuthology.orchestra.run.smithi059:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.1.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.1.crt -days 365 -sha256 2019-12-09T09:44:16.173 INFO:teuthology.orchestra.run.smithi059.stderr:Signature ok 2019-12-09T09:44:16.174 INFO:teuthology.orchestra.run.smithi059.stderr:subject=/CN=smithi161.front.sepia.ceph.com 2019-12-09T09:44:16.174 INFO:teuthology.orchestra.run.smithi059.stderr:Getting CA Private Key 2019-12-09T09:44:16.185 INFO:teuthology.orchestra.run.smithi059:> rm /home/ubuntu/cephtest/ca/rgw.client.1.csr /home/ubuntu/cephtest/ca/root.srl 2019-12-09T09:44:16.255 INFO:teuthology.orchestra.run.smithi059:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.1.crt 2019-12-09T09:44:16.320 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/ca/rgw.client.1.crt: OK 2019-12-09T09:44:16.321 INFO:tasks.openssl_keys:copying from ubuntu@smithi059.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.crt to ubuntu@smithi161.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.crt... 2019-12-09T09:44:16.384 DEBUG:teuthology.orchestra.remote:smithi059:/home/ubuntu/cephtest/ca/rgw.client.1.crt is 1KB 2019-12-09T09:44:16.537 INFO:tasks.openssl_keys:copying from ubuntu@smithi059.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.key to ubuntu@smithi161.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.key... 2019-12-09T09:44:16.616 DEBUG:teuthology.orchestra.remote:smithi059:/home/ubuntu/cephtest/ca/rgw.client.1.key is 2KB 2019-12-09T09:44:16.784 INFO:teuthology.orchestra.run.smithi059:> rm /home/ubuntu/cephtest/ca/rgw.client.1.crt /home/ubuntu/cephtest/ca/rgw.client.1.key 2019-12-09T09:44:16.847 INFO:teuthology.orchestra.run.smithi161:> openssl verify /home/ubuntu/cephtest/ca/rgw.client.1.crt 2019-12-09T09:44:16.962 INFO:teuthology.orchestra.run.smithi161.stdout:/home/ubuntu/cephtest/ca/rgw.client.1.crt: OK 2019-12-09T09:44:16.963 INFO:teuthology.orchestra.run.smithi161:> cat /home/ubuntu/cephtest/ca/rgw.client.1.key >> /home/ubuntu/cephtest/ca/rgw.client.1.crt 2019-12-09T09:44:17.041 INFO:teuthology.orchestra.run.smithi059:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-09T09:44:17.133 INFO:teuthology.orchestra.run.smithi059:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.0.key -out /home/ubuntu/cephtest/ca/rgw.client.0.csr -subj /CN=smithi059.front.sepia.ceph.com 2019-12-09T09:44:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:Generating a 2048 bit RSA private key 2019-12-09T09:44:17.252 INFO:teuthology.orchestra.run.smithi059.stderr:...............................+++ 2019-12-09T09:44:17.269 INFO:teuthology.orchestra.run.smithi059.stderr:...........+++ 2019-12-09T09:44:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:writing new private key to '/home/ubuntu/cephtest/ca/rgw.client.0.key' 2019-12-09T09:44:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:----- 2019-12-09T09:44:17.272 INFO:teuthology.orchestra.run.smithi059:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.0.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.0.crt -days 365 -sha256 2019-12-09T09:44:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:Signature ok 2019-12-09T09:44:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:subject=/CN=smithi059.front.sepia.ceph.com 2019-12-09T09:44:17.346 INFO:teuthology.orchestra.run.smithi059.stderr:Getting CA Private Key 2019-12-09T09:44:17.361 INFO:teuthology.orchestra.run.smithi059:> rm /home/ubuntu/cephtest/ca/rgw.client.0.csr /home/ubuntu/cephtest/ca/root.srl 2019-12-09T09:44:17.427 INFO:teuthology.orchestra.run.smithi059:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.0.crt 2019-12-09T09:44:17.492 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/ca/rgw.client.0.crt: OK 2019-12-09T09:44:17.493 INFO:teuthology.orchestra.run.smithi059:> cat /home/ubuntu/cephtest/ca/rgw.client.0.key >> /home/ubuntu/cephtest/ca/rgw.client.0.crt 2019-12-09T09:44:17.542 INFO:teuthology.run_tasks:Running task rgw... 2019-12-09T09:44:17.567 DEBUG:tasks.rgw:config is {'client.1': {'ssl certificate': 'rgw.client.1'}, 'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-12-09T09:44:17.567 DEBUG:tasks.rgw:client list is ['client.0'] 2019-12-09T09:44:17.567 INFO:tasks.rgw:Creating data pools 2019-12-09T09:44:17.567 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-12-09T09:44:17.568 INFO:teuthology.orchestra.run.smithi059:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2019-12-09T09:44:18.787 INFO:teuthology.orchestra.run.smithi059:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-12-09T09:44:21.324 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'default.rgw.buckets.data' created 2019-12-09T09:44:21.343 INFO:teuthology.orchestra.run.smithi059:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-12-09T09:44:22.787 INFO:teuthology.orchestra.run.smithi059.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-12-09T09:44:22.804 INFO:teuthology.orchestra.run.smithi059:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-12-09T09:44:24.017 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'default.rgw.buckets.index' created 2019-12-09T09:44:24.034 INFO:teuthology.orchestra.run.smithi059:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-12-09T09:44:26.027 INFO:teuthology.orchestra.run.smithi059.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-12-09T09:44:26.040 DEBUG:tasks.rgw:Pools created 2019-12-09T09:44:26.040 INFO:tasks.rgw:Configuring compression type = random 2019-12-09T09:44:26.040 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T09:44:26.041 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T09:44:26.041 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T09:44:26.198 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:26.199+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:28.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.630+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:28.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.630+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:28.663 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.663+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:28.663 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.663+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:28.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.681+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:28.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.681+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:28.814 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:28.808+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.522+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:30.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.532+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:30.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.532+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:30.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.538+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:30.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.538+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:30.546 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.546+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:30.546 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.546+0000 7f6655e76ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:44:30.547 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.546+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:30.556 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.556+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:30.556 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.556+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:30.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.565+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:30.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.565+0000 7f6655e76ec0 10 Creating default zonegroup 2019-12-09T09:44:30.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.565+0000 7f6655e76ec0 20 get_system_obj_state: rctx=0x7ffcf4b07c40 obj=:.pools.avail state=0x558bbbfae4d0 s->prefetch_data=0 2019-12-09T09:44:30.568 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.568+0000 7f6655e76ec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-12-09T09:44:30.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.758+0000 7f6655e76ec0 10 WARNING: store->list_zones() returned r=-2 2019-12-09T09:44:30.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.758+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:30.768+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.308+0000 7f66438a1700 0 -- 172.21.15.59:0/3841594449 >> [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] conn(0x7f663002c630 msgr2=0x7f663002fea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:44:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.730+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.738+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.738+0000 7f6655e76ec0 10 could not read realm id: (2) No such file or directory 2019-12-09T09:44:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.738+0000 7f6655e76ec0 10 WARNING: failed to set zone as default, r=-22 2019-12-09T09:44:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.738+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:31.744+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:32.393 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.393+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.447+0000 7f6655e76ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:44:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.447+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:32.512 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.512+0000 7f6655e76ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:44:32.512 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.512+0000 7f6655e76ec0 20 zonegroup default 2019-12-09T09:44:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.512+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:32.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.525+0000 7f6655e76ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:32.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.525+0000 7f6655e76ec0 10 Cannot find current period zone using local zone 2019-12-09T09:44:32.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.525+0000 7f6655e76ec0 10 Using default name default 2019-12-09T09:44:32.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.525+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.578+0000 7f6655e76ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.578+0000 7f6655e76ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.592+0000 7f6655e76ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:44:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.592+0000 7f6655e76ec0 20 zone default found 2019-12-09T09:44:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.592+0000 7f6655e76ec0 20 started sync module instance, tier type = 2019-12-09T09:44:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:32.592+0000 7f6655e76ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T09:44:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:35.354+0000 7f664289f700 0 -- 172.21.15.59:0/3841594449 >> [v2:172.21.15.161:6808/9940,v1:172.21.15.161:6809/9940] conn(0x558bbbfb6550 msgr2=0x558bbbfb8970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:44:36.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.711+0000 7f6655e76ec0 20 add_watcher() i=0 2019-12-09T09:44:36.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.711+0000 7f6655e76ec0 20 add_watcher() i=1 2019-12-09T09:44:36.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.711+0000 7f6655e76ec0 20 add_watcher() i=2 2019-12-09T09:44:36.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.711+0000 7f6655e76ec0 20 add_watcher() i=3 2019-12-09T09:44:36.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.711+0000 7f6655e76ec0 20 add_watcher() i=4 2019-12-09T09:44:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.711+0000 7f6655e76ec0 20 add_watcher() i=5 2019-12-09T09:44:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.743+0000 7f6655e76ec0 20 add_watcher() i=6 2019-12-09T09:44:36.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.767+0000 7f6655e76ec0 20 add_watcher() i=7 2019-12-09T09:44:36.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.767+0000 7f6655e76ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:44:36.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:36.767+0000 7f6613fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:44:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:40.933+0000 7f6655e76ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-12-09T09:44:45.094 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:45.094+0000 7f6655e76ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-12-09T09:44:49.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.059+0000 7f657e7fc700 20 reqs_thread_entry: start 2019-12-09T09:44:49.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.059+0000 7f656bfff700 20 reqs_thread_entry: start 2019-12-09T09:44:49.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.059+0000 7f6655e76ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:44:49.060 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.060+0000 7f656a7fc700 20 reqs_thread_entry: start 2019-12-09T09:44:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.358+0000 7f6655e76ec0 20 remove_watcher() i=0 2019-12-09T09:44:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.358+0000 7f6655e76ec0 2 removed watcher, disabling cache 2019-12-09T09:44:49.435 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.436+0000 7f6655e76ec0 20 remove_watcher() i=1 2019-12-09T09:44:49.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.525+0000 7f6655e76ec0 20 remove_watcher() i=2 2019-12-09T09:44:49.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.621+0000 7f6655e76ec0 20 remove_watcher() i=3 2019-12-09T09:44:49.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.703+0000 7f6655e76ec0 20 remove_watcher() i=4 2019-12-09T09:44:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.771+0000 7f6655e76ec0 20 remove_watcher() i=5 2019-12-09T09:44:49.869 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.869+0000 7f6655e76ec0 20 remove_watcher() i=6 2019-12-09T09:44:49.951 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:49.951+0000 7f6655e76ec0 20 remove_watcher() i=7 2019-12-09T09:44:49.963 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2019-12-09T09:44:49.963 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T09:44:49.963 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T09:44:49.963 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T09:44:49.964 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-12-09T09:44:50.220 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.220+0000 7fcc94ff9700 20 reqs_thread_entry: start 2019-12-09T09:44:50.220 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.220+0000 7fcd57c64ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.248+0000 7fcd57c64ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.248+0000 7fcd57c64ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.271+0000 7fcd57c64ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:44:50.272 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.271+0000 7fcd57c64ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.278+0000 7fcd57c64ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:50.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.278+0000 7fcd57c64ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.287+0000 7fcd57c64ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:50.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.287+0000 7fcd57c64ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.305+0000 7fcd57c64ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:44:50.534 INFO:teuthology.orchestra.run.smithi059.stdout:{"id":"478c2bee-74bb-4cd4-b56a-b2f2fa8233cf","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:44:50.535 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'478c2bee-74bb-4cd4-b56a-b2f2fa8233cf', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:44:50.535 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T09:44:50.535 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T09:44:50.536 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T09:44:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.751+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.794+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.794+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.807+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:50.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.807+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.816+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:50.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.816+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:50.844+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:51.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.204+0000 7f29fcdeaec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T09:44:51.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.204+0000 7f29fcdeaec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T09:44:51.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.204+0000 7f29fcdeaec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:44:51.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.204+0000 7f29fcdeaec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:44:51.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.204+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.226+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:51.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.226+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.246 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.246+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:44:51.246 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.246+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.284 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.284+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:51.284 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.284+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.305+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:44:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.305+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.315+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.315+0000 7f29fcdeaec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:44:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.315+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.328+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:51.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.328+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.343+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.343+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.363+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:44:51.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.363+0000 7f29fcdeaec0 20 zonegroup default 2019-12-09T09:44:51.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.363+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.374+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.374+0000 7f29fcdeaec0 10 Cannot find current period zone using local zone 2019-12-09T09:44:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.374+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.385 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.385+0000 7f29fcdeaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:51.385 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.385+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.402 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.402+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:51.402 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.402+0000 7f29fcdeaec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:51.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.417+0000 7f29fcdeaec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:44:51.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.417+0000 7f29fcdeaec0 20 zone default found 2019-12-09T09:44:51.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.417+0000 7f29fcdeaec0 20 started sync module instance, tier type = 2019-12-09T09:44:51.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.417+0000 7f29fcdeaec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T09:44:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=0 2019-12-09T09:44:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=1 2019-12-09T09:44:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=2 2019-12-09T09:44:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=3 2019-12-09T09:44:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=4 2019-12-09T09:44:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=5 2019-12-09T09:44:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.913+0000 7f29fcdeaec0 20 add_watcher() i=6 2019-12-09T09:44:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.952+0000 7f29fcdeaec0 20 add_watcher() i=7 2019-12-09T09:44:52.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.952+0000 7f29fcdeaec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:44:52.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.952+0000 7f29bafdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:44:52.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:51.956+0000 7f29e9813700 0 -- 172.21.15.59:0/1535947419 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55ab2a07d000 msgr2=0x55ab2a07f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:44:52.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:52.002+0000 7f29ea014700 0 -- 172.21.15.59:0/207159493 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55ab2a07f2e0 msgr2=0x55ab2a081700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:44:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:54.819+0000 7f29297fa700 20 reqs_thread_entry: start 2019-12-09T09:44:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:54.820+0000 7f29fcdeaec0 20 init_complete bucket index max shards: 0 2019-12-09T09:44:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:54.820+0000 7f2922ffd700 20 reqs_thread_entry: start 2019-12-09T09:44:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:54.820+0000 7f29217fa700 20 reqs_thread_entry: start 2019-12-09T09:44:55.110 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.110+0000 7f29fcdeaec0 20 remove_watcher() i=0 2019-12-09T09:44:55.110 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.110+0000 7f29fcdeaec0 2 removed watcher, disabling cache 2019-12-09T09:44:55.145 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.145+0000 7f29fcdeaec0 20 remove_watcher() i=1 2019-12-09T09:44:55.176 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.176+0000 7f29fcdeaec0 20 remove_watcher() i=2 2019-12-09T09:44:55.243 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.243+0000 7f29fcdeaec0 20 remove_watcher() i=3 2019-12-09T09:44:55.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.289+0000 7f29fcdeaec0 20 remove_watcher() i=4 2019-12-09T09:44:55.319 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.319+0000 7f29fcdeaec0 20 remove_watcher() i=5 2019-12-09T09:44:55.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.389+0000 7f29fcdeaec0 20 remove_watcher() i=6 2019-12-09T09:44:55.432 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.432+0000 7f29fcdeaec0 20 remove_watcher() i=7 2019-12-09T09:44:55.438 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2019-12-09T09:44:55.439 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T09:44:55.439 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-12-09T09:44:55.439 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T09:44:55.439 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T09:44:55.440 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-12-09T09:44:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.625+0000 7f5b717fa700 20 reqs_thread_entry: start 2019-12-09T09:44:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.625+0000 7f5c14306ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:55.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.643+0000 7f5c14306ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:55.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.643+0000 7f5c14306ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:55.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.667+0000 7f5c14306ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:55.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.667+0000 7f5c14306ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:55.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.682+0000 7f5c14306ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:44:55.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.682+0000 7f5c14306ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.704+0000 7f5c14306ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:44:55.797 INFO:teuthology.orchestra.run.smithi059.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-12-09T09:44:55.797 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T09:44:55.797 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T09:44:55.797 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T09:44:55.798 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-12-09T09:44:55.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.997+0000 7f22fdffb700 20 reqs_thread_entry: start 2019-12-09T09:44:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:55.997+0000 7f23c8653ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.017+0000 7f23c8653ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.017+0000 7f23c8653ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.040 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.040+0000 7f23c8653ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:44:56.040 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.040+0000 7f23c8653ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.051+0000 7f23c8653ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:56.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.051+0000 7f23c8653ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.070+0000 7f23c8653ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:56.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.070+0000 7f23c8653ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.095 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.095+0000 7f23c8653ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T09:44:56.169 INFO:teuthology.orchestra.run.smithi059.stdout:{"id":"478c2bee-74bb-4cd4-b56a-b2f2fa8233cf","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:44:56.170 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'478c2bee-74bb-4cd4-b56a-b2f2fa8233cf', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:44:56.170 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-12-09T09:44:56.170 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T09:44:56.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T09:44:56.171 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-12-09T09:44:56.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.389+0000 7f255b7fe700 20 reqs_thread_entry: start 2019-12-09T09:44:56.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.389+0000 7f26260f4ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.407 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.407+0000 7f26260f4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:56.407 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.407+0000 7f26260f4ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.434 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.434+0000 7f26260f4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:56.434 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.434+0000 7f26260f4ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.458+0000 7f26260f4ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T09:44:56.458 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.458+0000 7f26260f4ec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.484 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.481+0000 7f26260f4ec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T09:44:56.542 INFO:teuthology.orchestra.run.smithi059.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-12-09T09:44:56.542 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T09:44:56.543 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T09:44:56.543 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T09:44:56.543 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-12-09T09:44:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.770+0000 7f1094ff9700 20 reqs_thread_entry: start 2019-12-09T09:44:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.770+0000 7f115790cec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.794+0000 7f115790cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.794+0000 7f115790cec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.813+0000 7f115790cec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T09:44:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.813+0000 7f115790cec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.822 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.823+0000 7f115790cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:44:56.822 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.823+0000 7f115790cec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.835 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.836+0000 7f115790cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:44:56.836 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.836+0000 7f115790cec0 20 rados->read ofs=0 len=0 2019-12-09T09:44:56.860 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:44:56.860+0000 7f115790cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:44:56.935 INFO:teuthology.orchestra.run.smithi059.stdout:{"id":"478c2bee-74bb-4cd4-b56a-b2f2fa8233cf","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:44:56.936 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'478c2bee-74bb-4cd4-b56a-b2f2fa8233cf', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:44:56.937 INFO:tasks.rgw:Starting rgw... 2019-12-09T09:44:56.937 INFO:tasks.rgw:rgw client.0 config is {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-12-09T09:44:56.937 INFO:tasks.rgw:Using beast as radosgw frontend 2019-12-09T09:44:56.937 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-12-09T09:44:56.938 INFO:tasks.rgw.client.0:Restarting daemon 2019-12-09T09:44:56.938 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-12-09T09:44:56.942 INFO:tasks.rgw.client.0:Started 2019-12-09T09:44:56.942 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on https://smithi059.front.sepia.ceph.com:443/ 2019-12-09T09:44:56.942 INFO:teuthology.orchestra.run.smithi059:> curl https://smithi059.front.sepia.ceph.com:443/ 2019-12-09T09:44:57.034 INFO:teuthology.orchestra.run.smithi059.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:44:57.034 INFO:teuthology.orchestra.run.smithi059.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:44:57.037 INFO:teuthology.orchestra.run.smithi059.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi059.front.sepia.ceph.com:443; Connection refused 2019-12-09T09:44:57.038 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:44:58.039 INFO:teuthology.orchestra.run.smithi059:> curl https://smithi059.front.sepia.ceph.com:443/ 2019-12-09T09:44:58.127 INFO:teuthology.orchestra.run.smithi059.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:44:58.128 INFO:teuthology.orchestra.run.smithi059.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:44:58.132 INFO:teuthology.orchestra.run.smithi059.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi059.front.sepia.ceph.com:443; Connection refused 2019-12-09T09:44:58.132 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:45:00.133 INFO:teuthology.orchestra.run.smithi059:> curl https://smithi059.front.sepia.ceph.com:443/ 2019-12-09T09:45:00.205 INFO:teuthology.orchestra.run.smithi059.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:45:00.206 INFO:teuthology.orchestra.run.smithi059.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:45:00.210 INFO:teuthology.orchestra.run.smithi059.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi059.front.sepia.ceph.com:443; Connection refused 2019-12-09T09:45:00.210 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:45:04.213 INFO:teuthology.orchestra.run.smithi059:> curl https://smithi059.front.sepia.ceph.com:443/ 2019-12-09T09:45:04.300 INFO:teuthology.orchestra.run.smithi059.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:45:04.300 INFO:teuthology.orchestra.run.smithi059.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:45:04.307 INFO:teuthology.orchestra.run.smithi059.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi059.front.sepia.ceph.com:443; Connection refused 2019-12-09T09:45:04.308 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:45:12.309 INFO:teuthology.orchestra.run.smithi059:> curl https://smithi059.front.sepia.ceph.com:443/ 2019-12-09T09:45:12.378 INFO:teuthology.orchestra.run.smithi059.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:45:12.380 INFO:teuthology.orchestra.run.smithi059.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:45:13.938 INFO:teuthology.orchestra.run.smithi059.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 100 214 0 214 0 0 136 0 --:--:-- 0:00:01 --:--:-- 136 2019-12-09T09:45:13.939 INFO:teuthology.orchestra.run.smithi059.stdout:anonymous 2019-12-09T09:45:13.940 INFO:teuthology.run_tasks:Running task workunit... 2019-12-09T09:45:13.959 INFO:tasks.workunit:Pulling workunits from ref 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:45:13.959 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-12-09T09:45:13.959 INFO:teuthology.orchestra.run.smithi059:> stat -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:45:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-12-09T09:45:14.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:45:14.011 INFO:teuthology.orchestra.run.smithi059:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:45:14.059 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-12-09T09:45:14.060 INFO:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-12-09T09:45:14.118 INFO:tasks.workunit:timeout=3h 2019-12-09T09:45:14.118 INFO:tasks.workunit:cleanup=True 2019-12-09T09:45:14.119 INFO:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:45:14.406 INFO:tasks.workunit.client.0.smithi059.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-12-09T09:46:20.988 INFO:tasks.workunit.client.0.smithi059.stderr:Note: checking out '1351c819e1a9518cde4c8b12d645782009028360'. 2019-12-09T09:46:20.989 INFO:tasks.workunit.client.0.smithi059.stderr: 2019-12-09T09:46:20.989 INFO:tasks.workunit.client.0.smithi059.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-12-09T09:46:20.989 INFO:tasks.workunit.client.0.smithi059.stderr:changes and commit them, and you can discard any commits you make in this 2019-12-09T09:46:20.989 INFO:tasks.workunit.client.0.smithi059.stderr:state without impacting any branches by performing another checkout. 2019-12-09T09:46:20.989 INFO:tasks.workunit.client.0.smithi059.stderr: 2019-12-09T09:46:20.990 INFO:tasks.workunit.client.0.smithi059.stderr:If you want to create a new branch to retain commits you create, you may 2019-12-09T09:46:20.990 INFO:tasks.workunit.client.0.smithi059.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-12-09T09:46:20.990 INFO:tasks.workunit.client.0.smithi059.stderr: 2019-12-09T09:46:20.990 INFO:tasks.workunit.client.0.smithi059.stderr: git checkout -b new_branch_name 2019-12-09T09:46:20.990 INFO:tasks.workunit.client.0.smithi059.stderr: 2019-12-09T09:46:20.990 INFO:tasks.workunit.client.0.smithi059.stderr:HEAD is now at 1351c81... rgw: Fix dynamic resharding not working for empty zonegroup in period 2019-12-09T09:46:20.991 INFO:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-12-09T09:46:21.061 INFO:tasks.workunit.client.0.smithi059.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-12-09T09:46:21.064 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T09:46:21.064 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-12-09T09:46:21.520 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-12-09T09:46:21.616 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-12-09T09:46:21.658 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T09:46:21.659 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T09:46:21.660 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-12-09T09:46:21.698 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T09:46:21.745 DEBUG:teuthology.orchestra.remote:smithi059:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-12-09T09:46:21.794 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-12-09T09:46:21.794 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-12-09T09:46:21.795 INFO:teuthology.orchestra.run.smithi059:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-12-09T09:46:21.846 INFO:tasks.workunit.client.0.smithi059.stdout:Running main() from gmock_main.cc 2019-12-09T09:46:21.847 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] Running 11 tests from 1 test suite. 2019-12-09T09:46:21.847 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment set-up. 2019-12-09T09:46:21.847 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 11 tests from ClsLock 2019-12-09T09:46:21.847 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-12-09T09:46:24.593 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:24.593 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: exclusive 2019-12-09T09:46:24.593 INFO:tasks.workunit.client.0.smithi059.stdout: tag: 2019-12-09T09:46:24.593 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:24.594 INFO:tasks.workunit.client.0.smithi059.stdout: client.4476 expiration=2019-12-09T09:48:24.417675+0000 addr=v1:172.21.15.59:0/3538525224 cookie= 2019-12-09T09:46:24.794 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:24.794 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: shared 2019-12-09T09:46:24.794 INFO:tasks.workunit.client.0.smithi059.stdout: tag: 2019-12-09T09:46:24.794 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:24.795 INFO:tasks.workunit.client.0.smithi059.stdout: client.4476 expiration=2019-12-09T09:48:24.733069+0000 addr=v1:172.21.15.59:0/3538525224 cookie= 2019-12-09T09:46:24.795 INFO:tasks.workunit.client.0.smithi059.stdout: client.4516 expiration=0.000000 addr=v1:172.21.15.59:0/944705091 cookie= 2019-12-09T09:46:24.857 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:24.857 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: shared 2019-12-09T09:46:24.857 INFO:tasks.workunit.client.0.smithi059.stdout: tag: 2019-12-09T09:46:24.857 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:24.858 INFO:tasks.workunit.client.0.smithi059.stdout: client.4516 expiration=0.000000 addr=v1:172.21.15.59:0/944705091 cookie= 2019-12-09T09:46:26.179 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestMultiLocking (4332 ms) 2019-12-09T09:46:26.179 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestMeta 2019-12-09T09:46:28.996 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:28.996 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: shared 2019-12-09T09:46:28.996 INFO:tasks.workunit.client.0.smithi059.stdout: tag: 2019-12-09T09:46:28.997 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:28.997 INFO:tasks.workunit.client.0.smithi059.stdout: client.4376 expiration=0.000000 addr=v1:172.21.15.59:0/618500974 cookie= 2019-12-09T09:46:29.095 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:29.098 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: exclusive 2019-12-09T09:46:29.111 INFO:tasks.workunit.client.0.smithi059.stdout: tag: new_tag 2019-12-09T09:46:29.124 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:29.125 INFO:tasks.workunit.client.0.smithi059.stdout: client.4488 expiration=0.000000 addr=v1:172.21.15.59:0/3108369417 cookie= 2019-12-09T09:46:29.726 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestMeta (3547 ms) 2019-12-09T09:46:29.726 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestCookie 2019-12-09T09:46:32.639 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:32.639 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: exclusive 2019-12-09T09:46:32.639 INFO:tasks.workunit.client.0.smithi059.stdout: tag: 2019-12-09T09:46:32.639 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:32.726 INFO:tasks.workunit.client.0.smithi059.stdout:lock: mylock 2019-12-09T09:46:32.727 INFO:tasks.workunit.client.0.smithi059.stdout: lock_type: shared 2019-12-09T09:46:32.727 INFO:tasks.workunit.client.0.smithi059.stdout: tag: 2019-12-09T09:46:32.727 INFO:tasks.workunit.client.0.smithi059.stdout: lockers: 2019-12-09T09:46:32.727 INFO:tasks.workunit.client.0.smithi059.stdout: client.4497 expiration=0.000000 addr=v1:172.21.15.59:0/3351567750 cookie= 2019-12-09T09:46:32.727 INFO:tasks.workunit.client.0.smithi059.stdout: client.4497 expiration=0.000000 addr=v1:172.21.15.59:0/3351567750 cookie=new cookie 2019-12-09T09:46:33.069 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestCookie (3342 ms) 2019-12-09T09:46:33.069 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-12-09T09:46:36.472 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestMultipleLocks (3404 ms) 2019-12-09T09:46:36.472 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestLockDuration 2019-12-09T09:46:44.699 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestLockDuration (8227 ms) 2019-12-09T09:46:44.699 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-12-09T09:46:48.207 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestAssertLocked (3508 ms) 2019-12-09T09:46:48.207 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestSetCookie 2019-12-09T09:46:51.550 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestSetCookie (3344 ms) 2019-12-09T09:46:51.551 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestRenew 2019-12-09T09:47:12.969 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestRenew (21419 ms) 2019-12-09T09:47:12.969 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-12-09T09:47:20.891 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7921 ms) 2019-12-09T09:47:20.891 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-12-09T09:47:29.351 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8461 ms) 2019-12-09T09:47:29.352 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-12-09T09:47:37.233 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7882 ms) 2019-12-09T09:47:37.234 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 11 tests from ClsLock (75387 ms total) 2019-12-09T09:47:37.234 INFO:tasks.workunit.client.0.smithi059.stdout: 2019-12-09T09:47:37.234 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment tear-down 2019-12-09T09:47:37.234 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] 11 tests from 1 test suite ran. (75387 ms total) 2019-12-09T09:47:37.235 INFO:tasks.workunit.client.0.smithi059.stdout:[ PASSED ] 11 tests. 2019-12-09T09:47:37.236 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:47:37.236 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:47:37.345 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-12-09T09:47:37.345 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-12-09T09:47:37.345 INFO:teuthology.orchestra.run.smithi059:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-12-09T09:47:37.405 INFO:tasks.workunit.client.0.smithi059.stdout:Running main() from gmock_main.cc 2019-12-09T09:47:37.405 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] Running 4 tests from 1 test suite. 2019-12-09T09:47:37.406 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment set-up. 2019-12-09T09:47:37.406 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 4 tests from cls_log 2019-12-09T09:47:37.406 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-12-09T09:47:40.610 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_log.test_log_add_same_time (3204 ms) 2019-12-09T09:47:40.610 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-12-09T09:47:44.180 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_log.test_log_add_different_time (3571 ms) 2019-12-09T09:47:44.180 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_log.trim_by_time 2019-12-09T09:47:49.893 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_log.trim_by_time (5713 ms) 2019-12-09T09:47:49.893 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_log.trim_by_marker 2019-12-09T09:47:54.468 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_log.trim_by_marker (4575 ms) 2019-12-09T09:47:54.469 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 4 tests from cls_log (17063 ms total) 2019-12-09T09:47:54.469 INFO:tasks.workunit.client.0.smithi059.stdout: 2019-12-09T09:47:54.469 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment tear-down 2019-12-09T09:47:54.469 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] 4 tests from 1 test suite ran. (17063 ms total) 2019-12-09T09:47:54.469 INFO:tasks.workunit.client.0.smithi059.stdout:[ PASSED ] 4 tests. 2019-12-09T09:47:54.470 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:47:54.471 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:47:54.538 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-12-09T09:47:54.538 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-12-09T09:47:54.538 INFO:teuthology.orchestra.run.smithi059:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-12-09T09:47:54.592 INFO:tasks.workunit.client.0.smithi059.stdout:Running main() from gmock_main.cc 2019-12-09T09:47:54.592 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] Running 5 tests from 1 test suite. 2019-12-09T09:47:54.592 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment set-up. 2019-12-09T09:47:54.593 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 5 tests from cls_rgw 2019-12-09T09:47:54.593 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.test_implicit 2019-12-09T09:47:57.824 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.test_implicit (3232 ms) 2019-12-09T09:47:57.824 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-12-09T09:48:02.455 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.test_implicit_idempotent (4631 ms) 2019-12-09T09:48:02.455 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.test_put_snap 2019-12-09T09:48:10.727 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T09:48:10.689+0000 36a79700 -1 osd.5 pg_epoch: 106 pg[25.6( v 105'4 (0'0,105'4] local-lis/les=102/103 n=2 ec=102/102 lis/c=102/102 les/c/f=103/103/0 sis=102) [5,1] r=0 lpr=102 crt=104'3 lcod 104'3 mlcod 104'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-12-09T09:48:12.618 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.test_put_snap (10163 ms) 2019-12-09T09:48:12.618 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.test_explicit 2019-12-09T09:48:16.255 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.test_explicit (3637 ms) 2019-12-09T09:48:16.255 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.set 2019-12-09T09:48:19.689 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.set (3434 ms) 2019-12-09T09:48:19.689 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 5 tests from cls_rgw (25097 ms total) 2019-12-09T09:48:19.689 INFO:tasks.workunit.client.0.smithi059.stdout: 2019-12-09T09:48:19.689 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment tear-down 2019-12-09T09:48:19.689 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] 5 tests from 1 test suite ran. (25097 ms total) 2019-12-09T09:48:19.689 INFO:tasks.workunit.client.0.smithi059.stdout:[ PASSED ] 5 tests. 2019-12-09T09:48:19.691 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:48:19.691 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:48:19.759 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-12-09T09:48:19.760 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-12-09T09:48:19.760 INFO:teuthology.orchestra.run.smithi059:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-12-09T09:48:19.823 INFO:tasks.workunit.client.0.smithi059.stdout:Running main() from gmock_main.cc 2019-12-09T09:48:19.823 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] Running 13 tests from 1 test suite. 2019-12-09T09:48:19.823 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment set-up. 2019-12-09T09:48:19.823 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 13 tests from cls_rgw 2019-12-09T09:48:21.995 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.index_basic 2019-12-09T09:48:23.678 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.index_basic (1682 ms) 2019-12-09T09:48:23.678 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-12-09T09:48:25.051 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (1373 ms) 2019-12-09T09:48:25.051 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.index_remove_object 2019-12-09T09:48:27.317 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.index_remove_object (2266 ms) 2019-12-09T09:48:27.317 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.index_suggest 2019-12-09T09:48:54.380 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.index_suggest (27064 ms) 2019-12-09T09:48:54.380 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.index_list 2019-12-09T09:48:55.735 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.index_list (1355 ms) 2019-12-09T09:48:55.735 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.bi_list 2019-12-09T09:48:58.606 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.bi_list (2871 ms) 2019-12-09T09:48:58.606 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.gc_set 2019-12-09T09:48:59.119 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.gc_set (512 ms) 2019-12-09T09:48:59.119 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.gc_list 2019-12-09T09:48:59.616 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.gc_list (497 ms) 2019-12-09T09:48:59.616 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.gc_defer 2019-12-09T09:49:07.859 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.gc_defer (8244 ms) 2019-12-09T09:49:07.860 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.usage_basic 2019-12-09T09:49:16.598 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.usage_basic (8739 ms) 2019-12-09T09:49:16.598 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-12-09T09:49:16.634 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.usage_clear_no_obj (36 ms) 2019-12-09T09:49:16.634 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.usage_clear 2019-12-09T09:49:24.199 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.usage_clear (7565 ms) 2019-12-09T09:49:24.199 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-12-09T09:49:26.411 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] cls_rgw.bi_log_trim (2212 ms) 2019-12-09T09:49:27.111 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 13 tests from cls_rgw (64416 ms total) 2019-12-09T09:49:27.111 INFO:tasks.workunit.client.0.smithi059.stdout: 2019-12-09T09:49:27.111 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment tear-down 2019-12-09T09:49:27.112 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] 13 tests from 1 test suite ran. (67289 ms total) 2019-12-09T09:49:27.112 INFO:tasks.workunit.client.0.smithi059.stdout:[ PASSED ] 13 tests. 2019-12-09T09:49:27.113 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:49:27.114 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:49:27.232 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-12-09T09:49:27.232 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-12-09T09:49:27.232 INFO:teuthology.orchestra.run.smithi059:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-12-09T09:49:27.316 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] Running 6 tests from 1 test suite. 2019-12-09T09:49:27.316 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment set-up. 2019-12-09T09:49:29.436 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 6 tests from rgw_gc_log 2019-12-09T09:49:29.437 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-12-09T09:49:29.917 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] rgw_gc_log.init_existing_queue (480 ms) 2019-12-09T09:49:29.917 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-12-09T09:49:31.280 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] rgw_gc_log.init_existing_omap (1363 ms) 2019-12-09T09:49:31.280 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-12-09T09:49:31.423 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (144 ms) 2019-12-09T09:49:31.424 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-12-09T09:49:31.513 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (90 ms) 2019-12-09T09:49:31.513 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-12-09T09:49:31.597 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] rgw_gc_log.defer1_after_init (83 ms) 2019-12-09T09:49:31.597 INFO:tasks.workunit.client.0.smithi059.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-12-09T09:49:31.640 INFO:tasks.workunit.client.0.smithi059.stdout:[ OK ] rgw_gc_log.defer2_before_init (43 ms) 2019-12-09T09:49:31.640 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] 6 tests from rgw_gc_log (2203 ms total) 2019-12-09T09:49:31.640 INFO:tasks.workunit.client.0.smithi059.stdout: 2019-12-09T09:49:31.640 INFO:tasks.workunit.client.0.smithi059.stdout:[----------] Global test environment tear-down 2019-12-09T09:49:32.751 INFO:tasks.workunit.client.0.smithi059.stdout:[==========] 6 tests from 1 test suite ran. (5436 ms total) 2019-12-09T09:49:32.751 INFO:tasks.workunit.client.0.smithi059.stdout:[ PASSED ] 6 tests. 2019-12-09T09:49:32.754 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:49:32.754 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:49:32.822 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-12-09T09:49:32.822 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-12-09T09:49:33.070 DEBUG:teuthology.parallel:result is None 2019-12-09T09:49:33.071 INFO:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:49:33.143 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:49:33.144 INFO:teuthology.orchestra.run.smithi059:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:49:33.199 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:49:33.199 INFO:teuthology.run_tasks:Running task ragweed... 2019-12-09T09:49:33.212 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-12-09T09:49:33.214 INFO:tasks.ragweed:Downloading ragweed... 2019-12-09T09:49:33.214 INFO:tasks.ragweed:Using branch 'wip-ofriedma-dynamic-disable' for ragweed 2019-12-09T09:49:33.214 INFO:teuthology.orchestra.run.smithi059:> git clone -b wip-ofriedma-dynamic-disable git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T09:49:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:fatal: Remote branch wip-ofriedma-dynamic-disable not found in upstream origin 2019-12-09T09:49:33.275 INFO:teuthology.orchestra.run.smithi059.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T09:49:33.275 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-12-09T09:49:33.275 INFO:teuthology.orchestra.run.smithi059:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T09:49:33.368 INFO:teuthology.orchestra.run.smithi059.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T09:49:33.462 INFO:tasks.ragweed:Creating rgw users... 2019-12-09T09:49:33.462 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-12-09T09:49:33.464 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key WUKGAQURULVJJJOYZCGI --secret k0LtRsbDqYeMihDHHSdy5dg000jC1kYwd2elhzW3ncERAMg/YX7hEg== --email sysuser.client.0+test@test.test --system true 2019-12-09T09:49:33.614 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.617+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.639 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.641+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.640 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.641+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.646+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.646+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.651+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.651+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.663+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.884+0000 7fefc0e5bec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T09:49:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.884+0000 7fefc0e5bec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T09:49:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.884+0000 7fefc0e5bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:49:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.884+0000 7fefc0e5bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:49:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.884+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.897+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:33.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.897+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.907+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:49:33.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.907+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.918 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.921+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:33.919 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.921+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.933+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:49:33.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.933+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.941+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.941+0000 7fefc0e5bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:49:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.941+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.951+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.951+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.959 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.961+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:33.959 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.961+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.971+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:49:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.971+0000 7fefc0e5bec0 20 zonegroup default 2019-12-09T09:49:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.971+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.976+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.976+0000 7fefc0e5bec0 10 Cannot find current period zone using local zone 2019-12-09T09:49:33.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.976+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.985+0000 7fefc0e5bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:33.983 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.985+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:33.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.993+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:33.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:33.993+0000 7fefc0e5bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.007+0000 7fefc0e5bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:49:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.007+0000 7fefc0e5bec0 20 zone default found 2019-12-09T09:49:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.007+0000 7fefc0e5bec0 20 started sync module instance, tier type = 2019-12-09T09:49:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.007+0000 7fefc0e5bec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T09:49:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=0 2019-12-09T09:49:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=1 2019-12-09T09:49:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=2 2019-12-09T09:49:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=3 2019-12-09T09:49:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=4 2019-12-09T09:49:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=5 2019-12-09T09:49:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.334+0000 7fefc0e5bec0 20 add_watcher() i=6 2019-12-09T09:49:34.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.375+0000 7fefc0e5bec0 20 add_watcher() i=7 2019-12-09T09:49:34.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.375+0000 7fefc0e5bec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:49:34.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.375+0000 7fef7efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:49:34.375 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.377+0000 7fefae085700 0 -- 172.21.15.59:0/3418726442 >> v1:172.21.15.59:6789/0 conn(0x55f4ee844060 legacy=0x55f4ee8464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:49:34.405 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:34.408+0000 7fefae886700 0 -- 172.21.15.59:0/2990222883 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feef00036d0 msgr2=0x7feef0003ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:49:36.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.184+0000 7feeed7fa700 20 reqs_thread_entry: start 2019-12-09T09:49:36.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.184+0000 7feee6ffd700 20 reqs_thread_entry: start 2019-12-09T09:49:36.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.184+0000 7fefc0e5bec0 20 init_complete bucket index max shards: 0 2019-12-09T09:49:36.183 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.185+0000 7feee57fa700 20 reqs_thread_entry: start 2019-12-09T09:49:36.183 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.185+0000 7fefc0e5bec0 20 get_system_obj_state: rctx=0x7ffc1caf40e8 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x55f4ee8bb710 s->prefetch_data=0 2019-12-09T09:49:36.183 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.185+0000 7fefc0e5bec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T09:49:36.198 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.201+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-12-09T09:49:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.201+0000 7fefc0e5bec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:49:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.201+0000 7fefc0e5bec0 20 get_system_obj_state: rctx=0x7ffc1caf40f8 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x55f4ee8bd4a0 s->prefetch_data=0 2019-12-09T09:49:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.201+0000 7fefc0e5bec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-12-09T09:49:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.210+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-12-09T09:49:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.210+0000 7fefc0e5bec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:49:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.210+0000 7fefc0e5bec0 20 get_system_obj_state: rctx=0x7ffc1caf40f8 obj=default.rgw.meta:users.keys:WUKGAQURULVJJJOYZCGI state=0x55f4ee8bd4a0 s->prefetch_data=0 2019-12-09T09:49:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.210+0000 7fefc0e5bec0 10 cache get: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI : miss 2019-12-09T09:49:36.216 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.217+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI info.flags=0x0 2019-12-09T09:49:36.217 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.217+0000 7fefc0e5bec0 10 adding default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI to cache LRU end 2019-12-09T09:49:36.217 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.217+0000 7fefc0e5bec0 20 get_system_obj_state: rctx=0x7ffc1caf3d08 obj=default.rgw.meta:users.keys:WUKGAQURULVJJJOYZCGI state=0x55f4ee8bd4a0 s->prefetch_data=0 2019-12-09T09:49:36.217 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.217+0000 7fefc0e5bec0 10 cache get: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI : type miss (requested=0x6, cached=0x0) 2019-12-09T09:49:36.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.223+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI info.flags=0x0 2019-12-09T09:49:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.223+0000 7fefc0e5bec0 10 moving default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI to cache LRU end 2019-12-09T09:49:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.223+0000 7fefc0e5bec0 20 get_system_obj_state: rctx=0x7ffc1caf3e88 obj=default.rgw.meta:users.keys:WUKGAQURULVJJJOYZCGI state=0x55f4ee8bd4a0 s->prefetch_data=0 2019-12-09T09:49:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.223+0000 7fefc0e5bec0 10 cache get: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI : type miss (requested=0x6, cached=0x0) 2019-12-09T09:49:36.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.230+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI info.flags=0x0 2019-12-09T09:49:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.230+0000 7fefc0e5bec0 10 moving default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI to cache LRU end 2019-12-09T09:49:36.298 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.301+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T09:49:36.299 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.301+0000 7fefc0e5bec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:49:36.299 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.301+0000 7fefc0e5bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-12-09T09:49:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.370+0000 7fef8f7fe700 10 RGWWatcher::handle_notify() notify_id 545460846592 cookie 94510461950048 notifier 4666 bl.length()=637 2019-12-09T09:49:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.370+0000 7fef8f7fe700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T09:49:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:36.370+0000 7fef8f7fe700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:49:37.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.354+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T09:49:37.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.354+0000 7fefc0e5bec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:49:37.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.354+0000 7fefc0e5bec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-12-09T09:49:37.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.373+0000 7fef8f7fe700 10 RGWWatcher::handle_notify() notify_id 545460846593 cookie 94510461955456 notifier 4666 bl.length()=202 2019-12-09T09:49:37.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.373+0000 7fef8f7fe700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T09:49:37.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.373+0000 7fef8f7fe700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:49:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.483+0000 7fefc0e5bec0 10 cache put: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI info.flags=0x7 2019-12-09T09:49:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.483+0000 7fefc0e5bec0 10 moving default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI to cache LRU end 2019-12-09T09:49:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.483+0000 7fefc0e5bec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=190 2019-12-09T09:49:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.522+0000 7fef8f7fe700 10 RGWWatcher::handle_notify() notify_id 545460846592 cookie 94510461961424 notifier 4666 bl.length()=190 2019-12-09T09:49:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.522+0000 7fef8f7fe700 10 cache put: name=default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI info.flags=0x7 2019-12-09T09:49:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.522+0000 7fef8f7fe700 10 moving default.rgw.meta+users.keys+WUKGAQURULVJJJOYZCGI to cache LRU end 2019-12-09T09:49:37.674 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T09:49:37.674 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "sysuser.client.0", 2019-12-09T09:49:37.674 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. sysuser.client.0", 2019-12-09T09:49:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "sysuser.client.0+test@test.test", 2019-12-09T09:49:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T09:49:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T09:49:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [], 2019-12-09T09:49:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [ 2019-12-09T09:49:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "sysuser.client.0", 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: "access_key": "WUKGAQURULVJJJOYZCGI", 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "k0LtRsbDqYeMihDHHSdy5dg000jC1kYwd2elhzW3ncERAMg/YX7hEg==" 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [], 2019-12-09T09:49:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "system": "true", 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T09:49:37.677 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T09:49:37.678 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T09:49:37.679 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T09:49:37.680 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T09:49:37.680 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.715+0000 7fefc0e5bec0 20 remove_watcher() i=0 2019-12-09T09:49:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.715+0000 7fefc0e5bec0 2 removed watcher, disabling cache 2019-12-09T09:49:37.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.760+0000 7fefc0e5bec0 20 remove_watcher() i=1 2019-12-09T09:49:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.819+0000 7fefc0e5bec0 20 remove_watcher() i=2 2019-12-09T09:49:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.876+0000 7fefc0e5bec0 20 remove_watcher() i=3 2019-12-09T09:49:37.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.933+0000 7fefc0e5bec0 20 remove_watcher() i=4 2019-12-09T09:49:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:37.986+0000 7fefc0e5bec0 20 remove_watcher() i=5 2019-12-09T09:49:38.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.039+0000 7fefc0e5bec0 20 remove_watcher() i=6 2019-12-09T09:49:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.102+0000 7fefc0e5bec0 20 remove_watcher() i=7 2019-12-09T09:49:38.108 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-12-09T09:49:38.109 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key SIPGHYIHMETDGLIVBURF --secret E611enZXL+FNnIHz0UfUgyj9eHEwLNQcGDV9JBaBn2ytEWYrgzhlqw== --email ragweed.client.0+test@test.test --system false 2019-12-09T09:49:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.254+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.282+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.280 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.282+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.289+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.289+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.297+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.297+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.318+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.537+0000 7f7b8110bec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T09:49:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.537+0000 7f7b8110bec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T09:49:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.537+0000 7f7b8110bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:49:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.537+0000 7f7b8110bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:49:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.537+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.549 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.551+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:38.549 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.551+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.568+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:49:38.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.568+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.586+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.587+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.605+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:49:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.605+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.614+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.614+0000 7f7b8110bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:49:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.614+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.620 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.623+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.620 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.623+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.634+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:38.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.634+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.649+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:49:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.649+0000 7f7b8110bec0 20 zonegroup default 2019-12-09T09:49:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.649+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.659+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.659+0000 7f7b8110bec0 10 Cannot find current period zone using local zone 2019-12-09T09:49:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.659+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.667+0000 7f7b8110bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:49:38.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.667+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.680 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.683+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:49:38.681 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.683+0000 7f7b8110bec0 20 rados->read ofs=0 len=0 2019-12-09T09:49:38.699 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.701+0000 7f7b8110bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:49:38.699 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.701+0000 7f7b8110bec0 20 zone default found 2019-12-09T09:49:38.699 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.701+0000 7f7b8110bec0 20 started sync module instance, tier type = 2019-12-09T09:49:38.700 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:38.701+0000 7f7b8110bec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T09:49:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=0 2019-12-09T09:49:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=1 2019-12-09T09:49:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=2 2019-12-09T09:49:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=3 2019-12-09T09:49:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=4 2019-12-09T09:49:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=5 2019-12-09T09:49:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.057+0000 7f7b8110bec0 20 add_watcher() i=6 2019-12-09T09:49:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.094+0000 7f7b8110bec0 20 add_watcher() i=7 2019-12-09T09:49:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.094+0000 7f7b8110bec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:49:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.094+0000 7f7b3efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:49:39.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.095+0000 7f7b6db34700 0 -- >> v1:172.21.15.161:6789/0 conn(0x563efee5ec90 legacy=0x563efee611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:49:39.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:39.136+0000 7f7b6e335700 0 -- 172.21.15.59:0/1333630107 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7f7ab0022470 msgr2=0x7f7ab0024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:49:40.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.577+0000 7f7aad7fa700 20 reqs_thread_entry: start 2019-12-09T09:49:40.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.577+0000 7f7aa6ffd700 20 reqs_thread_entry: start 2019-12-09T09:49:40.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.577+0000 7f7b8110bec0 20 init_complete bucket index max shards: 0 2019-12-09T09:49:40.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.578+0000 7f7aa57fa700 20 reqs_thread_entry: start 2019-12-09T09:49:40.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.578+0000 7f7b8110bec0 20 get_system_obj_state: rctx=0x7ffc38a12658 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x563efeed97b0 s->prefetch_data=0 2019-12-09T09:49:40.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.578+0000 7f7b8110bec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T09:49:40.585 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.587+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-12-09T09:49:40.585 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.587+0000 7f7b8110bec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:49:40.585 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.587+0000 7f7b8110bec0 20 get_system_obj_state: rctx=0x7ffc38a12668 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x563efeedb540 s->prefetch_data=0 2019-12-09T09:49:40.585 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.587+0000 7f7b8110bec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-12-09T09:49:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.594+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-12-09T09:49:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.594+0000 7f7b8110bec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:49:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.594+0000 7f7b8110bec0 20 get_system_obj_state: rctx=0x7ffc38a12668 obj=default.rgw.meta:users.keys:SIPGHYIHMETDGLIVBURF state=0x563efeedb540 s->prefetch_data=0 2019-12-09T09:49:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.594+0000 7f7b8110bec0 10 cache get: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF : miss 2019-12-09T09:49:40.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.605+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF info.flags=0x0 2019-12-09T09:49:40.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.605+0000 7f7b8110bec0 10 adding default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF to cache LRU end 2019-12-09T09:49:40.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.605+0000 7f7b8110bec0 20 get_system_obj_state: rctx=0x7ffc38a12278 obj=default.rgw.meta:users.keys:SIPGHYIHMETDGLIVBURF state=0x563efeedb540 s->prefetch_data=0 2019-12-09T09:49:40.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.605+0000 7f7b8110bec0 10 cache get: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF : type miss (requested=0x6, cached=0x0) 2019-12-09T09:49:40.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.616+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF info.flags=0x0 2019-12-09T09:49:40.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.616+0000 7f7b8110bec0 10 moving default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF to cache LRU end 2019-12-09T09:49:40.614 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.616+0000 7f7b8110bec0 20 get_system_obj_state: rctx=0x7ffc38a123f8 obj=default.rgw.meta:users.keys:SIPGHYIHMETDGLIVBURF state=0x563efeedb540 s->prefetch_data=0 2019-12-09T09:49:40.614 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.616+0000 7f7b8110bec0 10 cache get: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF : type miss (requested=0x6, cached=0x0) 2019-12-09T09:49:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.625+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF info.flags=0x0 2019-12-09T09:49:40.623 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.625+0000 7f7b8110bec0 10 moving default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF to cache LRU end 2019-12-09T09:49:40.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.685+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T09:49:40.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.685+0000 7f7b8110bec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:49:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.685+0000 7f7b8110bec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-12-09T09:49:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.697+0000 7f7b4f7fe700 10 RGWWatcher::handle_notify() notify_id 545460846594 cookie 94828564353760 notifier 4629 bl.length()=637 2019-12-09T09:49:40.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.697+0000 7f7b4f7fe700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T09:49:40.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:40.697+0000 7f7b4f7fe700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:49:41.563 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.565+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T09:49:41.563 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.565+0000 7f7b8110bec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:49:41.563 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.565+0000 7f7b8110bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-12-09T09:49:41.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.582+0000 7f7b4f7fe700 10 RGWWatcher::handle_notify() notify_id 545460846595 cookie 94828564352192 notifier 4629 bl.length()=202 2019-12-09T09:49:41.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.583+0000 7f7b4f7fe700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T09:49:41.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.583+0000 7f7b4f7fe700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:49:41.942 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.944+0000 7f7b8110bec0 10 cache put: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF info.flags=0x7 2019-12-09T09:49:41.942 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.944+0000 7f7b8110bec0 10 moving default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF to cache LRU end 2019-12-09T09:49:41.942 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.944+0000 7f7b8110bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=190 2019-12-09T09:49:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.965+0000 7f7b4f7fe700 10 RGWWatcher::handle_notify() notify_id 545460846596 cookie 94828564352192 notifier 4629 bl.length()=190 2019-12-09T09:49:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.965+0000 7f7b4f7fe700 10 cache put: name=default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF info.flags=0x7 2019-12-09T09:49:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:41.965+0000 7f7b4f7fe700 10 moving default.rgw.meta+users.keys+SIPGHYIHMETDGLIVBURF to cache LRU end 2019-12-09T09:49:42.235 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T09:49:42.236 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "ragweed.client.0", 2019-12-09T09:49:42.236 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. ragweed.client.0", 2019-12-09T09:49:42.236 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "ragweed.client.0+test@test.test", 2019-12-09T09:49:42.236 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T09:49:42.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T09:49:42.236 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [], 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [ 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "ragweed.client.0", 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: "access_key": "SIPGHYIHMETDGLIVBURF", 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "E611enZXL+FNnIHz0UfUgyj9eHEwLNQcGDV9JBaBn2ytEWYrgzhlqw==" 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T09:49:42.237 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [], 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "system": "true", 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T09:49:42.238 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T09:49:42.239 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T09:49:42.239 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T09:49:42.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T09:49:42.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T09:49:42.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T09:49:42.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T09:49:42.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T09:49:42.241 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T09:49:42.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T09:49:42.241 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T09:49:42.241 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T09:49:42.241 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.288+0000 7f7b8110bec0 20 remove_watcher() i=0 2019-12-09T09:49:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.288+0000 7f7b8110bec0 2 removed watcher, disabling cache 2019-12-09T09:49:42.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.359+0000 7f7b8110bec0 20 remove_watcher() i=1 2019-12-09T09:49:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.420+0000 7f7b8110bec0 20 remove_watcher() i=2 2019-12-09T09:49:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.490+0000 7f7b8110bec0 20 remove_watcher() i=3 2019-12-09T09:49:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.552+0000 7f7b8110bec0 20 remove_watcher() i=4 2019-12-09T09:49:42.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.601+0000 7f7b8110bec0 20 remove_watcher() i=5 2019-12-09T09:49:42.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.658+0000 7f7b8110bec0 20 remove_watcher() i=6 2019-12-09T09:49:42.701 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T09:49:42.703+0000 7f7b8110bec0 20 remove_watcher() i=7 2019-12-09T09:49:42.709 INFO:tasks.ragweed:Configuring ragweed... 2019-12-09T09:49:42.709 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-12-09T09:49:49.060 INFO:teuthology.orchestra.run.smithi059.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T09:49:49.217 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:49:49.286 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:49:49.287 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:49:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:49:49.288 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:49:49.289 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T09:49:49.948 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:49:50.298 INFO:teuthology.orchestra.run.smithi059.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-12-09T09:49:51.179 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T09:49:51.179 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:49:51.179 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-12-09T09:49:51.180 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-12-09T09:49:51.183 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-12-09T09:49:51.221 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-12-09T09:49:51.222 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-12-09T09:49:51.223 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-12-09T09:49:51.223 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:49:51.223 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-12-09T09:49:51.224 INFO:teuthology.orchestra.run.smithi059.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-12-09T09:49:51.227 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-12-09T09:49:51.228 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T09:49:51.228 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-12-09T09:49:51.623 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T09:49:51.820 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:51.821 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T09:49:51.821 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:51.822 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:49:51.822 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T09:49:51.822 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:49:51.822 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T09:49:51.822 INFO:teuthology.orchestra.run.smithi059.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout:Installing for dependencies: 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-12-09T09:49:51.823 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:51.824 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T09:49:51.824 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T09:49:51.824 INFO:teuthology.orchestra.run.smithi059.stdout:Install 4 Packages (+3 Dependent packages) 2019-12-09T09:49:51.824 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:51.825 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 1.7 M 2019-12-09T09:49:51.825 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 12 M 2019-12-09T09:49:51.825 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T09:49:52.739 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:49:52.739 INFO:teuthology.orchestra.run.smithi059.stdout:Total 1.8 MB/s | 1.7 MB 00:00 2019-12-09T09:49:52.748 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T09:49:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T09:49:52.821 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T09:49:52.822 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T09:49:53.445 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-12-09T09:49:53.723 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-12-09T09:49:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T09:49:54.701 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-12-09T09:49:55.160 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-12-09T09:49:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-12-09T09:49:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-12-09T09:49:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-12-09T09:49:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-12-09T09:49:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T09:49:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-12-09T09:49:56.304 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-12-09T09:49:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-12-09T09:49:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout:Dependency Installed: 2019-12-09T09:49:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-12-09T09:49:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-12-09T09:49:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:49:56.876 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T09:49:57.117 INFO:teuthology.orchestra.run.smithi059.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-12-09T09:49:59.015 INFO:teuthology.orchestra.run.smithi059.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T09:49:59.591 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting pip 2019-12-09T09:49:59.692 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-12-09T09:49:59.903 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: pip 2019-12-09T09:49:59.904 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: pip 9.0.1 2019-12-09T09:49:59.966 INFO:teuthology.orchestra.run.smithi059.stdout: Uninstalling pip-9.0.1: 2019-12-09T09:50:00.008 INFO:teuthology.orchestra.run.smithi059.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T09:50:00.616 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed pip-19.3.1 2019-12-09T09:50:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:50:01.501 INFO:teuthology.orchestra.run.smithi059.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-12-09T09:50:02.495 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:50:02.707 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting nose 2019-12-09T09:50:02.749 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-12-09T09:50:02.878 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: nose 2019-12-09T09:50:02.960 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed nose-1.3.7 2019-12-09T09:50:03.539 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:50:04.175 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting setuptools==32.3.1 2019-12-09T09:50:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-12-09T09:50:04.434 INFO:teuthology.orchestra.run.smithi059.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T09:50:04.434 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: setuptools 2019-12-09T09:50:04.434 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T09:50:04.454 INFO:teuthology.orchestra.run.smithi059.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T09:50:04.470 INFO:teuthology.orchestra.run.smithi059.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T09:50:04.597 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed setuptools-32.3.1 2019-12-09T09:50:05.116 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:50:05.402 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting PyYAML 2019-12-09T09:50:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB) 2019-12-09T09:50:06.141 INFO:teuthology.orchestra.run.smithi059.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-12-09T09:50:06.306 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-12-09T09:50:06.316 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-12-09T09:50:06.621 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting bunch>=1.0.0 2019-12-09T09:50:06.631 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-12-09T09:50:07.412 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting gevent>=1.0 2019-12-09T09:50:07.434 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-12-09T09:50:07.998 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting httplib2 2019-12-09T09:50:08.013 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-12-09T09:50:09.349 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting lxml 2019-12-09T09:50:09.365 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-12-09T09:50:09.840 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T09:50:09.851 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T09:50:09.914 INFO:teuthology.orchestra.run.smithi059.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-12-09T09:50:09.915 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for PyYAML (setup.py): started 2019-12-09T09:50:10.607 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-12-09T09:50:10.607 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for PyYAML: filename=PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl size=46299 sha256=8e6ba696b1eece23878aa73291afd208c1504cc4d4bc5e19fc41a5a22d2b32d5 2019-12-09T09:50:10.607 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67 2019-12-09T09:50:10.623 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for bunch (setup.py): started 2019-12-09T09:50:10.946 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-12-09T09:50:10.947 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=717173e31897123261eb97306d5810cc9a0720f1dc049bc509b93b1497254529 2019-12-09T09:50:10.947 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-12-09T09:50:10.949 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for httplib2 (setup.py): started 2019-12-09T09:50:11.266 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-12-09T09:50:11.266 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=24647a2c7aaaaaa6ea42772c4960b21c0c898d7a5d9e72247ab9ceebcaa417e1 2019-12-09T09:50:11.267 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-12-09T09:50:11.271 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully built PyYAML bunch httplib2 2019-12-09T09:50:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T09:50:11.375 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-12-09T09:50:11.375 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: PyYAML 3.10 2019-12-09T09:50:11.376 INFO:teuthology.orchestra.run.smithi059.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T09:50:11.376 INFO:teuthology.orchestra.run.smithi059.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-12-09T09:50:12.579 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: lxml 3.2.1 2019-12-09T09:50:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T09:50:12.581 INFO:teuthology.orchestra.run.smithi059.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-12-09T09:50:12.683 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.2 2019-12-09T09:50:13.181 INFO:teuthology.orchestra.run.smithi059.stdout:running develop 2019-12-09T09:50:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:running egg_info 2019-12-09T09:50:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:creating ragweed.egg-info 2019-12-09T09:50:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-12-09T09:50:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:writing ragweed.egg-info/PKG-INFO 2019-12-09T09:50:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-12-09T09:50:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-12-09T09:50:13.250 INFO:teuthology.orchestra.run.smithi059.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:50:13.250 INFO:teuthology.orchestra.run.smithi059.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:50:13.251 INFO:teuthology.orchestra.run.smithi059.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:50:13.252 INFO:teuthology.orchestra.run.smithi059.stdout:running build_ext 2019-12-09T09:50:13.253 INFO:teuthology.orchestra.run.smithi059.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-12-09T09:50:13.253 INFO:teuthology.orchestra.run.smithi059.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-12-09T09:50:13.253 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.254 INFO:teuthology.orchestra.run.smithi059.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-12-09T09:50:13.254 INFO:teuthology.orchestra.run.smithi059.stdout:Processing dependencies for ragweed==0.0.1 2019-12-09T09:50:13.258 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for isodate>=0.4.4 2019-12-09T09:50:13.259 INFO:teuthology.orchestra.run.smithi059.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-12-09T09:50:13.348 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-12-09T09:50:13.404 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: isodate 0.6.0 2019-12-09T09:50:13.405 INFO:teuthology.orchestra.run.smithi059.stdout:Processing isodate-0.6.0.tar.gz 2019-12-09T09:50:13.413 INFO:teuthology.orchestra.run.smithi059.stdout:Writing /tmp/easy_install-BX5dtT/isodate-0.6.0/setup.cfg 2019-12-09T09:50:13.413 INFO:teuthology.orchestra.run.smithi059.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-BX5dtT/isodate-0.6.0/egg-dist-tmp-uHLFsg 2019-12-09T09:50:13.705 INFO:teuthology.orchestra.run.smithi059.stderr:zip_safe flag not set; analyzing archive contents... 2019-12-09T09:50:13.722 INFO:teuthology.orchestra.run.smithi059.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:50:13.724 INFO:teuthology.orchestra.run.smithi059.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T09:50:13.726 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.726 INFO:teuthology.orchestra.run.smithi059.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-12-09T09:50:13.756 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for gevent==1.4.0 2019-12-09T09:50:13.756 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: gevent 1.4.0 2019-12-09T09:50:13.757 INFO:teuthology.orchestra.run.smithi059.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T09:50:13.757 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.758 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:50:13.758 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for bunch==1.0.1 2019-12-09T09:50:13.758 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: bunch 1.0.1 2019-12-09T09:50:13.758 INFO:teuthology.orchestra.run.smithi059.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T09:50:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.758 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:50:13.759 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for PyYAML==5.2 2019-12-09T09:50:13.760 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: PyYAML 5.2 2019-12-09T09:50:13.760 INFO:teuthology.orchestra.run.smithi059.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T09:50:13.760 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.760 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:50:13.760 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for boto==2.49.0 2019-12-09T09:50:13.760 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: boto 2.49.0 2019-12-09T09:50:13.761 INFO:teuthology.orchestra.run.smithi059.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T09:50:13.761 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.761 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:50:13.762 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for six==1.9.0 2019-12-09T09:50:13.762 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: six 1.9.0 2019-12-09T09:50:13.762 INFO:teuthology.orchestra.run.smithi059.stdout:Adding six 1.9.0 to easy-install.pth file 2019-12-09T09:50:13.762 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.762 INFO:teuthology.orchestra.run.smithi059.stdout:Using /usr/lib/python2.7/site-packages 2019-12-09T09:50:13.770 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for greenlet==0.4.15 2019-12-09T09:50:13.770 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: greenlet 0.4.15 2019-12-09T09:50:13.770 INFO:teuthology.orchestra.run.smithi059.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T09:50:13.771 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T09:50:13.771 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:50:13.772 INFO:teuthology.orchestra.run.smithi059.stdout:Finished processing dependencies for ragweed==0.0.1 2019-12-09T09:50:13.791 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-12-09T09:50:13.847 INFO:tasks.ragweed:Configuring boto... 2019-12-09T09:50:13.847 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T09:50:13.938 INFO:teuthology.orchestra.run.smithi059:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-12-09T09:51:27.155 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-12-09T09:52:15.788 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:09:12.403 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:15.888 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:09:12.515 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:15.897 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:09:12.524 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:15.953 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:09:12.565 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:16.008 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:09:12.635 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:16.052 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:09:12.664 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:16.613 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:09:13.240 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:16.738 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:09:13.337 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:16.859 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:09:13.486 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:52:41.116 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-12-09T09:53:38.914 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-12-09T09:54:31.193 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:11:27.821 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.201 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:11:27.830 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.213 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:11:27.842 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.215 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:11:27.829 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.216 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:11:27.846 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.219 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:11:27.833 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.330 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:11:27.959 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.338 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:11:27.967 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:31.349 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:11:27.962 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:51.363 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-12-09T09:56:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-12-09T09:59:11.595 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-12-09T09:59:32.255 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:16:28.887 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.297 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:16:28.928 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.316 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:16:28.948 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.328 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:16:28.961 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.345 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:16:28.965 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.347 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:16:28.967 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.486 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:16:29.118 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.518 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:16:29.150 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T09:59:32.523 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:16:29.143 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.300 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:17:33.932 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.325 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:17:33.958 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.346 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:17:33.978 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.347 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:17:33.980 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.377 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:17:33.998 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.393 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:17:34.026 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.406 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:17:34.027 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.414 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:17:34.047 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:37.466 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:17:34.088 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:43.625 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-12-09T10:04:43.166 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:21:39.795 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.167 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:21:39.799 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.181 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:21:39.814 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.204 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:21:39.819 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.206 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:21:39.822 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.241 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:21:39.874 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.309 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:21:39.942 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.370 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:21:39.986 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:04:43.401 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:21:40.033 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:07:02.448 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-12-09T10:08:09.386 INFO:teuthology.orchestra.run.smithi059.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-12-09T10:08:09.387 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T10:08:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:---------------------------------------------------------------------- 2019-12-09T10:08:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:Ran 9 tests in 1075.031s 2019-12-09T10:08:09.388 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T10:08:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:OK 2019-12-09T10:08:09.407 INFO:tasks.ragweed:Cleaning up boto... 2019-12-09T10:08:09.408 INFO:teuthology.orchestra.run.smithi059:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T10:08:09.471 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-12-09T10:08:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.618+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.650+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.650+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.656+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.656+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.666+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.666+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.674 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.683+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:09.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.892+0000 7fddde55dec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:08:09.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.892+0000 7fddde55dec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:08:09.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.892+0000 7fddde55dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:08:09.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.892+0000 7fddde55dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:08:09.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.892+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.915+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:09.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.915+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.934+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.934+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.949+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:09.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.949+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.951 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.960+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:08:09.951 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.960+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.959 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.968+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:09.959 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.968+0000 7fddde55dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:08:09.959 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.968+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.975+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.975+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.976 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.985+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:09.976 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.985+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.994+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:08:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.994+0000 7fddde55dec0 20 zonegroup default 2019-12-09T10:08:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:09.994+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:09.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.002+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.002+0000 7fddde55dec0 10 Cannot find current period zone using local zone 2019-12-09T10:08:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.002+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:10.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.013+0000 7fddde55dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:10.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.013+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.029+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.029+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.042+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:08:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.042+0000 7fddde55dec0 20 zone default found 2019-12-09T10:08:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.042+0000 7fddde55dec0 20 started sync module instance, tier type = 2019-12-09T10:08:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.042+0000 7fddde55dec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:08:10.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=0 2019-12-09T10:08:10.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=1 2019-12-09T10:08:10.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=2 2019-12-09T10:08:10.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=3 2019-12-09T10:08:10.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=4 2019-12-09T10:08:10.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=5 2019-12-09T10:08:10.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.377+0000 7fddde55dec0 20 add_watcher() i=6 2019-12-09T10:08:10.399 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.408+0000 7fddde55dec0 20 add_watcher() i=7 2019-12-09T10:08:10.399 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.408+0000 7fddde55dec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:08:10.399 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.408+0000 7fdd9bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:08:10.402 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.412+0000 7fddcbf88700 0 -- 172.21.15.59:0/3701746390 >> v2:172.21.15.161:3300/0 conn(0x55a99dcbc690 msgr2=0x55a99dcbcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:08:10.454 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:10.463+0000 7fddcb787700 0 -- 172.21.15.59:0/3022719621 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x55a99dcd5f50 msgr2=0x55a99dcd8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:08:12.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.309+0000 7fdd0a7fc700 20 reqs_thread_entry: start 2019-12-09T10:08:12.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.309+0000 7fdd03fff700 20 reqs_thread_entry: start 2019-12-09T10:08:12.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.309+0000 7fddde55dec0 20 init_complete bucket index max shards: 0 2019-12-09T10:08:12.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.309+0000 7fdd027fc700 20 reqs_thread_entry: start 2019-12-09T10:08:12.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.309+0000 7fddde55dec0 20 get_system_obj_state: rctx=0x7ffd1e9e9918 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x55a99dd3f550 s->prefetch_data=0 2019-12-09T10:08:12.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.309+0000 7fddde55dec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T10:08:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.325+0000 7fddde55dec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-12-09T10:08:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.325+0000 7fddde55dec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T10:08:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.325+0000 7fddde55dec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.325+0000 7fddde55dec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.325+0000 7fddde55dec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:08:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.325+0000 7fddde55dec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:12.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.347+0000 7fddde55dec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T10:08:12.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.347+0000 7fddde55dec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-12-09T10:08:12.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.347+0000 7fddde55dec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T10:08:12.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.353+0000 7fddde55dec0 10 removing key index: WUKGAQURULVJJJOYZCGI 2019-12-09T10:08:12.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.353+0000 7fddde55dec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=170 2019-12-09T10:08:12.355 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:12.363+0000 7fddc0ff9700 10 RGWWatcher::handle_notify() notify_id 584115552257 cookie 94186985159712 notifier 4653 bl.length()=170 2019-12-09T10:08:13.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.036+0000 7fddde55dec0 10 removing email index: sysuser.client.0+test@test.test 2019-12-09T10:08:13.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.036+0000 7fddde55dec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-12-09T10:08:13.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.055+0000 7fddc0ff9700 10 RGWWatcher::handle_notify() notify_id 584115552261 cookie 94186985151888 notifier 4653 bl.length()=182 2019-12-09T10:08:13.569 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.577+0000 7fddde55dec0 10 removing user buckets index 2019-12-09T10:08:13.569 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.577+0000 7fddde55dec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-12-09T10:08:13.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.611+0000 7fddc0ff9700 10 RGWWatcher::handle_notify() notify_id 584115552256 cookie 94186985153744 notifier 4653 bl.length()=173 2019-12-09T10:08:13.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.787+0000 7fddde55dec0 10 removing user index: sysuser.client.0 2019-12-09T10:08:13.780 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.787+0000 7fddde55dec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-12-09T10:08:13.780 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.787+0000 7fddde55dec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-12-09T10:08:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:13.810+0000 7fddc0ff9700 10 RGWWatcher::handle_notify() notify_id 584115552262 cookie 94186985150096 notifier 4653 bl.length()=165 2019-12-09T10:08:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.316+0000 7fddde55dec0 20 remove_watcher() i=0 2019-12-09T10:08:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.316+0000 7fddde55dec0 2 removed watcher, disabling cache 2019-12-09T10:08:14.377 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.384+0000 7fddde55dec0 20 remove_watcher() i=1 2019-12-09T10:08:14.435 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.443+0000 7fddde55dec0 20 remove_watcher() i=2 2019-12-09T10:08:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.501+0000 7fddde55dec0 20 remove_watcher() i=3 2019-12-09T10:08:14.551 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.559+0000 7fddde55dec0 20 remove_watcher() i=4 2019-12-09T10:08:14.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.624+0000 7fddde55dec0 20 remove_watcher() i=5 2019-12-09T10:08:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.699+0000 7fddde55dec0 20 remove_watcher() i=6 2019-12-09T10:08:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.751+0000 7fddde55dec0 20 remove_watcher() i=7 2019-12-09T10:08:14.752 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-12-09T10:08:14.901 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.909+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:14.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.942+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.942+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:14.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.952+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:14.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.952+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.961+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.961+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:14.986+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:15.184 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.192+0000 7f5294b78ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:08:15.185 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.192+0000 7f5294b78ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:08:15.185 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.192+0000 7f5294b78ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:08:15.185 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.192+0000 7f5294b78ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:08:15.185 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.192+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.196 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.204+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:15.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.204+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.217+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:08:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.217+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.219 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.226+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:15.219 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.227+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.235 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.243+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:08:15.235 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.243+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.241 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.249+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:15.241 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.249+0000 7f5294b78ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:08:15.242 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.249+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.258+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.258+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.274+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.274+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.290+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:08:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.290+0000 7f5294b78ec0 20 zonegroup default 2019-12-09T10:08:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.290+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.301+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.301+0000 7f5294b78ec0 10 Cannot find current period zone using local zone 2019-12-09T10:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.301+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.312+0000 7f5294b78ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.312+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.326+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.326+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.340+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.340+0000 7f5294b78ec0 20 zone default found 2019-12-09T10:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.340+0000 7f5294b78ec0 20 started sync module instance, tier type = 2019-12-09T10:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.340+0000 7f5294b78ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:08:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.521+0000 7f52815a1700 0 -- 172.21.15.59:0/1841238954 >> [v2:172.21.15.161:6808/9940,v1:172.21.15.161:6809/9940] conn(0x55c6b9321820 msgr2=0x55c6b9323ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:08:15.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=0 2019-12-09T10:08:15.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=1 2019-12-09T10:08:15.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=2 2019-12-09T10:08:15.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=3 2019-12-09T10:08:15.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=4 2019-12-09T10:08:15.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=5 2019-12-09T10:08:15.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.736+0000 7f5294b78ec0 20 add_watcher() i=6 2019-12-09T10:08:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.762+0000 7f5294b78ec0 20 add_watcher() i=7 2019-12-09T10:08:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.762+0000 7f5294b78ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:08:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.763+0000 7f52527dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:08:15.763 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:15.771+0000 7f5281da2700 0 -- 172.21.15.59:0/1066744149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c6b93324c0 msgr2=0x55c6b93388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:08:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.496+0000 7f51c8ff9700 20 reqs_thread_entry: start 2019-12-09T10:08:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.496+0000 7f51c27fc700 20 reqs_thread_entry: start 2019-12-09T10:08:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.496+0000 7f5294b78ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:08:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.497+0000 7f51c0ff9700 20 reqs_thread_entry: start 2019-12-09T10:08:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.497+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605448 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x55c6b93b39b0 s->prefetch_data=0 2019-12-09T10:08:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.497+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T10:08:17.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.515+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-12-09T10:08:17.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.515+0000 7f5294b78ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T10:08:17.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.515+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:17.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.515+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:17.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.515+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:08:17.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.515+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:17.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.534+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T10:08:17.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.534+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-12-09T10:08:17.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.534+0000 7f5294b78ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T10:08:17.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.566+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-conf state=0x55c6b93b52d0 s->prefetch_data=0 2019-12-09T10:08:17.559 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.566+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-12-09T10:08:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.582+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-12-09T10:08:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.582+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T10:08:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.582+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.582+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.582+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.582+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.592+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=200 2019-12-09T10:08:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.592+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-12-09T10:08:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.592+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T10:08:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.593+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 state=0x55c6b93b7890 s->prefetch_data=0 2019-12-09T10:08:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.593+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 : miss 2019-12-09T10:08:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 info.flags=0x16 2019-12-09T10:08:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 to cache LRU end 2019-12-09T10:08:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:17.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:17.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:17.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.622+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:17.650 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.657+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=292 2019-12-09T10:08:17.650 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.657+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 info.flags=0x13 2019-12-09T10:08:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.657+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 to cache LRU end 2019-12-09T10:08:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.657+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.657+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.657+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.681+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-conf[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1]) start [] num_entries 1100 2019-12-09T10:08:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-12-09T10:08:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-12-09T10:08:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-12-09T10:08:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-12-09T10:08:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-12-09T10:08:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-12-09T10:08:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-12-09T10:08:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-12-09T10:08:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-12-09T10:08:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.728+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.749+0000 7f5294b78ec0 10 manifest: total_size = 5851 2019-12-09T10:08:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.749+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.13 2019-12-09T10:08:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.749+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.749+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.749+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:17.918 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.925+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.947+0000 7f5294b78ec0 10 manifest: total_size = 3917 2019-12-09T10:08:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.947+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.30 2019-12-09T10:08:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.947+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.947+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:17.947+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:18.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.197+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.212+0000 7f5294b78ec0 10 manifest: total_size = 5681 2019-12-09T10:08:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.212+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.56 2019-12-09T10:08:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.212+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.212+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.213+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:18.413 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.420+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.440 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.448+0000 7f5294b78ec0 10 manifest: total_size = 5721 2019-12-09T10:08:18.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.448+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.72 2019-12-09T10:08:18.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.448+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.448+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:18.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.448+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.575+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.599+0000 7f5294b78ec0 10 manifest: total_size = 3924 2019-12-09T10:08:18.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.599+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.87 2019-12-09T10:08:18.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.599+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.599+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:18.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.599+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:18.816 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.823+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.835+0000 7f5294b78ec0 10 manifest: total_size = 12920 2019-12-09T10:08:18.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.835+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.120 2019-12-09T10:08:18.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.835+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:18.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.835+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:18.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:18.835+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.024+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.034+0000 7f5294b78ec0 10 manifest: total_size = 11062 2019-12-09T10:08:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.034+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.189 2019-12-09T10:08:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.034+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.034+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.034+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.200+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:19.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.214+0000 7f5294b78ec0 10 manifest: total_size = 12956 2019-12-09T10:08:19.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.214+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.246 2019-12-09T10:08:19.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.214+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:19.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.214+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:19.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.214+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.413+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.427+0000 7f5294b78ec0 10 manifest: total_size = 6764 2019-12-09T10:08:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.427+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.322 2019-12-09T10:08:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.427+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.427+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.427+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 2019-12-09T10:08:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.619+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-conf[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1]) start _multipart_[] num_entries 1001 2019-12-09T10:08:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.619+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:19.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.702+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.759+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-conf state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.759+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-12-09T10:08:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T10:08:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:19.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:19.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-12-09T10:08:19.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.775+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-12-09T10:08:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.807+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552256 cookie 94311998964688 notifier 4493 bl.length()=162 2019-12-09T10:08:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.987+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.1 from cache 2019-12-09T10:08:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:19.987+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=219 2019-12-09T10:08:19.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.000+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552257 cookie 94311998959280 notifier 4493 bl.length()=219 2019-12-09T10:08:20.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.701+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x55c6b93b52d0 s->prefetch_data=0 2019-12-09T10:08:20.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.701+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-12-09T10:08:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.720+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-12-09T10:08:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.720+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T10:08:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.720+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.720+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.720+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.720+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.735+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=229 2019-12-09T10:08:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.735+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-12-09T10:08:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.735+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T10:08:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.735+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 state=0x55c6b93bc2d0 s->prefetch_data=0 2019-12-09T10:08:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.735+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 : miss 2019-12-09T10:08:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 info.flags=0x16 2019-12-09T10:08:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 to cache LRU end 2019-12-09T10:08:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.757+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:20.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.780+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=321 2019-12-09T10:08:20.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.780+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 info.flags=0x13 2019-12-09T10:08:20.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.780+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 to cache LRU end 2019-12-09T10:08:20.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.780+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:20.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.780+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:20.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.780+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 2019-12-09T10:08:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.794+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2]) start [] num_entries 1100 2019-12-09T10:08:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.812+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T10:08:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.812+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.837+0000 7f5294b78ec0 10 manifest: total_size = 74198 2019-12-09T10:08:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.837+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.20 2019-12-09T10:08:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.837+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55c6b93bec38 s->prefetch_data=0 2019-12-09T10:08:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.837+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bec38 2019-12-09T10:08:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.837+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 2019-12-09T10:08:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T10:08:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=20613 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4210821 part_ofs=20613 rule->part_size=20613 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4231434 part_ofs=41226 rule->part_size=20613 2019-12-09T10:08:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-12-09T10:08:20.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-12-09T10:08:20.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4252047 part_ofs=61839 rule->part_size=12359 2019-12-09T10:08:20.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-12-09T10:08:20.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:20.975+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.20 2019-12-09T10:08:21.067 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.074+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2]) start _multipart_[] num_entries 1001 2019-12-09T10:08:21.067 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.074+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:21.085 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.090+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:21.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.169+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x55c6b93bfc20 s->prefetch_data=0 2019-12-09T10:08:21.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.169+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:21.178 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-12-09T10:08:21.178 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T10:08:21.178 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:21.178 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:21.178 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-12-09T10:08:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.185+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-12-09T10:08:21.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.201+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552258 cookie 94311998968000 notifier 4493 bl.length()=191 2019-12-09T10:08:21.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.846+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2 from cache 2019-12-09T10:08:21.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.846+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=248 2019-12-09T10:08:21.848 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:21.855+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552257 cookie 94311998964688 notifier 4493 bl.length()=248 2019-12-09T10:08:22.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.763+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x55c6b93b52d0 s->prefetch_data=0 2019-12-09T10:08:22.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.763+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-12-09T10:08:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.785+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-12-09T10:08:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.785+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T10:08:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.785+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.785+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.785+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.785+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.802+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=236 2019-12-09T10:08:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.802+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-12-09T10:08:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.802+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T10:08:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.802+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 state=0x55c6b93c4210 s->prefetch_data=0 2019-12-09T10:08:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.802+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 : miss 2019-12-09T10:08:22.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 info.flags=0x16 2019-12-09T10:08:22.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 to cache LRU end 2019-12-09T10:08:22.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.826+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:22.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.849+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=328 2019-12-09T10:08:22.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.849+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 info.flags=0x13 2019-12-09T10:08:22.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.849+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 to cache LRU end 2019-12-09T10:08:22.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.849+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:22.842 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.849+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:22.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.849+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 2019-12-09T10:08:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.872+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3]) start [] num_entries 1100 2019-12-09T10:08:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.903+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T10:08:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.903+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.914+0000 7f5294b78ec0 10 manifest: total_size = 26084 2019-12-09T10:08:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.914+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.40 2019-12-09T10:08:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.914+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.914+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93b9c78 2019-12-09T10:08:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:22.914+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 2019-12-09T10:08:23.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=4 2019-12-09T10:08:23.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=7674 2019-12-09T10:08:23.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=7674 stripe_ofs=7674 part_ofs=7674 rule->part_size=7671 2019-12-09T10:08:23.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=4 2019-12-09T10:08:23.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4197882 part_ofs=7674 rule->part_size=7671 2019-12-09T10:08:23.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=15345 stripe_ofs=15345 part_ofs=15345 rule->part_size=7672 2019-12-09T10:08:23.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=4 2019-12-09T10:08:23.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4205553 part_ofs=15345 rule->part_size=7672 2019-12-09T10:08:23.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=23017 stripe_ofs=23017 part_ofs=23017 rule->part_size=3067 2019-12-09T10:08:23.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2019-12-09T10:08:23.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4213225 part_ofs=23017 rule->part_size=3067 2019-12-09T10:08:23.060 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=26084 stripe_ofs=26084 part_ofs=26084 rule->part_size=3067 2019-12-09T10:08:23.060 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.065+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.40 2019-12-09T10:08:23.144 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.152+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3]) start _multipart_[] num_entries 1001 2019-12-09T10:08:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.152+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:23.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.168+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:23.213 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.221+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x55c6b93bc510 s->prefetch_data=0 2019-12-09T10:08:23.213 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.221+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-12-09T10:08:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T10:08:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:23.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:23.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-12-09T10:08:23.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.236+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-12-09T10:08:23.240 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.248+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552263 cookie 94311998969904 notifier 4493 bl.length()=198 2019-12-09T10:08:23.536 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.544+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3 from cache 2019-12-09T10:08:23.536 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.544+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=255 2019-12-09T10:08:23.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.558+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552258 cookie 94311998971296 notifier 4493 bl.length()=255 2019-12-09T10:08:23.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.979+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x55c6b93b52d0 s->prefetch_data=0 2019-12-09T10:08:23.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:23.980+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-12-09T10:08:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.011+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-12-09T10:08:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.011+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T10:08:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.011+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.011+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.011+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.011+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:24.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.019+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=231 2019-12-09T10:08:24.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.019+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-12-09T10:08:24.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.019+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T10:08:24.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.019+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 state=0x55c6b93bc510 s->prefetch_data=0 2019-12-09T10:08:24.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.019+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 : miss 2019-12-09T10:08:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 info.flags=0x16 2019-12-09T10:08:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 to cache LRU end 2019-12-09T10:08:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:24.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.035+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.055+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=323 2019-12-09T10:08:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.055+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 info.flags=0x13 2019-12-09T10:08:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.055+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 to cache LRU end 2019-12-09T10:08:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.055+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.055+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.055+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.068+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4]) start [] num_entries 1100 2019-12-09T10:08:24.080 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.088+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo[oJGr4.7112aOAelfLfFYXy4HXQ4Yt4l] 2019-12-09T10:08:24.080 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.088+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.096 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.105+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.133+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to x7ifg0ydy3nbkr9o16zwyjzta16gnsuj 2019-12-09T10:08:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.133+0000 7f5294b78ec0 20 get_obj_state_impl: setting s->olh_tag to mvuce3ewo4xkte3zbccxyla943h5eemc 2019-12-09T10:08:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.133+0000 7f5294b78ec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=mvuce3ewo4xkte3zbccxyla943h5eemc 2019-12-09T10:08:24.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.222+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.292 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.300+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.292 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.300+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.313+0000 7f5294b78ec0 20 olh_log_entry: epoch=3 op=2 key=[] 2019-12-09T10:08:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.313+0000 7f5294b78ec0 20 olh_log_entry: epoch=3 op=3 key=foo[oJGr4.7112aOAelfLfFYXy4HXQ4Yt4l] 2019-12-09T10:08:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.313+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-index-versioning.1:_:oJGr4.7112aOAelfLfFYXy4HXQ4Yt4l_foo state=0x55c6b93ba2e8 s->prefetch_data=0 2019-12-09T10:08:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.331+0000 7f5294b78ec0 10 manifest: total_size = 1848 2019-12-09T10:08:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.331+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.55 2019-12-09T10:08:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.331+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-index-versioning.1:_:oJGr4.7112aOAelfLfFYXy4HXQ4Yt4l_foo state=0x55c6b93ba2e8 s->prefetch_data=0 2019-12-09T10:08:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.331+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93ba2e8 2019-12-09T10:08:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.331+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:24.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=515 2019-12-09T10:08:24.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T10:08:24.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190723 part_ofs=515 rule->part_size=515 2019-12-09T10:08:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-09T10:08:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4191238 part_ofs=1030 rule->part_size=515 2019-12-09T10:08:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-09T10:08:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-09T10:08:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4191753 part_ofs=1545 rule->part_size=303 2019-12-09T10:08:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-09T10:08:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.463+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.55 2019-12-09T10:08:24.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.617+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.617+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.741+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.796+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 2019-12-09T10:08:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.796+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4]) start _multipart_[] num_entries 1001 2019-12-09T10:08:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.796+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:24.798 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.806+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:24.847 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.855+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x55c6b93c3f20 s->prefetch_data=0 2019-12-09T10:08:24.847 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.855+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-12-09T10:08:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T10:08:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-12-09T10:08:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.866+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-12-09T10:08:24.870 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:24.878+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552264 cookie 94311998969904 notifier 4493 bl.length()=193 2019-12-09T10:08:25.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.309+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4 from cache 2019-12-09T10:08:25.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.309+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=250 2019-12-09T10:08:25.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.327+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552259 cookie 94311998968000 notifier 4493 bl.length()=250 2019-12-09T10:08:25.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.744+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55c6b93bc510 s->prefetch_data=0 2019-12-09T10:08:25.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.744+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-12-09T10:08:25.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.761+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-12-09T10:08:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.761+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T10:08:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.761+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.761+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:25.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.761+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:25.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.761+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.774+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T10:08:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.774+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-12-09T10:08:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.774+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T10:08:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.774+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.774+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 : miss 2019-12-09T10:08:25.784 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 info.flags=0x16 2019-12-09T10:08:25.784 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 to cache LRU end 2019-12-09T10:08:25.784 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.792+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:25.803 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.811+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T10:08:25.803 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.811+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 info.flags=0x13 2019-12-09T10:08:25.803 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.811+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 to cache LRU end 2019-12-09T10:08:25.804 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.811+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:25.804 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.811+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:25.804 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.811+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 2019-12-09T10:08:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.828+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5]) start [] num_entries 1100 2019-12-09T10:08:25.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.845+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T10:08:25.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.845+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:25.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.863+0000 7f5294b78ec0 10 manifest: total_size = 1848 2019-12-09T10:08:25.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.863+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.71 2019-12-09T10:08:25.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.863+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:25.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.863+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93b9c78 2019-12-09T10:08:25.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:25.863+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 2019-12-09T10:08:26.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=515 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190723 part_ofs=515 rule->part_size=515 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:26.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4191238 part_ofs=1030 rule->part_size=515 2019-12-09T10:08:26.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-09T10:08:26.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-09T10:08:26.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4191753 part_ofs=1545 rule->part_size=303 2019-12-09T10:08:26.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-09T10:08:26.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.012+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.71 2019-12-09T10:08:26.073 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.081+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5]) start _multipart_[] num_entries 1001 2019-12-09T10:08:26.073 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.081+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:26.088 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.092+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:26.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.143+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:26.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.143+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-12-09T10:08:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T10:08:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-12-09T10:08:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.160+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-12-09T10:08:26.165 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.173+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552265 cookie 94311998957296 notifier 4493 bl.length()=183 2019-12-09T10:08:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.856+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5 from cache 2019-12-09T10:08:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.856+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=240 2019-12-09T10:08:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:26.867+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552257 cookie 94311998955056 notifier 4493 bl.length()=240 2019-12-09T10:08:27.504 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.512+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:27.504 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.512+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-12-09T10:08:27.521 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.529+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-12-09T10:08:27.521 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.529+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T10:08:27.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.529+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:27.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.529+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:27.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.529+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:27.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.529+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:27.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.546+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=227 2019-12-09T10:08:27.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.546+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-12-09T10:08:27.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.546+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T10:08:27.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.546+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 state=0x55c6b93bc510 s->prefetch_data=0 2019-12-09T10:08:27.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.546+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 : miss 2019-12-09T10:08:27.553 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 info.flags=0x16 2019-12-09T10:08:27.553 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 to cache LRU end 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:27.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.561+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.577+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=319 2019-12-09T10:08:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.577+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 info.flags=0x13 2019-12-09T10:08:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.577+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 to cache LRU end 2019-12-09T10:08:27.570 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.577+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:27.570 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.577+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:27.570 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.577+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:27.595 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.603+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7]) start [] num_entries 1100 2019-12-09T10:08:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T10:08:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T10:08:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T10:08:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T10:08:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T10:08:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T10:08:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.640+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.663+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.663+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.108 2019-12-09T10:08:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.663+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.663+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93b9c78 2019-12-09T10:08:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.663+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:27.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.824+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:27.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.824+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:27.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.824+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:27.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.824+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:27.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.824+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.108 2019-12-09T10:08:27.912 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.920+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:27.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.938+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:27.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.938+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.109 2019-12-09T10:08:27.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.938+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:27.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.938+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93b9c78 2019-12-09T10:08:27.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:27.938+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:28.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.071+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.109 2019-12-09T10:08:28.119 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.127+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.143+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.143+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _k1JMn1zJyDc-kOeoHxKX8qhNLpdqIn5 2019-12-09T10:08:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.143+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:28.136 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.143+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93b9c78 2019-12-09T10:08:28.136 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.143+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:28.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.254+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.254+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.254+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.254+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.248 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.254+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: _k1JMn1zJyDc-kOeoHxKX8qhNLpdqIn5 2019-12-09T10:08:28.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.332+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:28.336 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.344+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:28.336 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.344+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _iEPmserpwZTTurzmB3Xta6cR_-osotZ 2019-12-09T10:08:28.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.344+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:28.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.344+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:28.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.344+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:28.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.469+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.469+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.469+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: _iEPmserpwZTTurzmB3Xta6cR_-osotZ 2019-12-09T10:08:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.563+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:28.571 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.578+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:28.571 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.578+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.112 2019-12-09T10:08:28.571 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.578+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:28.571 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.578+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:28.571 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.578+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:28.686 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.693+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.686 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.693+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.686 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.693+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.686 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.693+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.693+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.112 2019-12-09T10:08:28.798 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.806+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.814+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.814+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _zwu9znG7QsMdQ-v_KAywQMR2wQEjenU 2019-12-09T10:08:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.814+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.814+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.814+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:28.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.944+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.944+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.944+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.944+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.944+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _zwu9znG7QsMdQ-v_KAywQMR2wQEjenU 2019-12-09T10:08:28.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:28.996+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.008+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.008+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.114 2019-12-09T10:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.008+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.008+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.008+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:29.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.120+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:29.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.121+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:29.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.121+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:29.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.121+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:29.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.121+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.114 2019-12-09T10:08:29.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.182+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.190 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.195+0000 7f5294b78ec0 10 manifest: total_size = 5242880 2019-12-09T10:08:29.190 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.195+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _Uev7uLtV9J03fd4YcqyPzib0NDCxpqa 2019-12-09T10:08:29.190 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.195+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.191 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.195+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:29.191 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.195+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:29.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.296+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:29.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.296+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T10:08:29.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.296+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: _Uev7uLtV9J03fd4YcqyPzib0NDCxpqa 2019-12-09T10:08:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.371+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.381 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.388+0000 7f5294b78ec0 10 manifest: total_size = 22651 2019-12-09T10:08:29.382 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.388+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.116 2019-12-09T10:08:29.382 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.388+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.382 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.388+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:29.382 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.388+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.491+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:29.496 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.503+0000 7f5294b78ec0 10 manifest: total_size = 22651 2019-12-09T10:08:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.503+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _8LIMi255n3XkXkIOeRYuloHVBCEERte 2019-12-09T10:08:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.503+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55c6b93b9c78 s->prefetch_data=0 2019-12-09T10:08:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.503+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93b9c78 2019-12-09T10:08:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.503+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:29.627 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.634+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:29.627 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.634+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2019-12-09T10:08:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.634+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: _8LIMi255n3XkXkIOeRYuloHVBCEERte 2019-12-09T10:08:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.703+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.714+0000 7f5294b78ec0 10 manifest: total_size = 22651 2019-12-09T10:08:29.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.714+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _lG2CWeqiMfNS73VW7jdCmY-VnzWVa2E 2019-12-09T10:08:29.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.714+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.714+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.714+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:29.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.825+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:29.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.825+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2019-12-09T10:08:29.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.825+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: _lG2CWeqiMfNS73VW7jdCmY-VnzWVa2E 2019-12-09T10:08:29.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.874+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.884+0000 7f5294b78ec0 10 manifest: total_size = 22651 2019-12-09T10:08:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.884+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _ZjlN5EsTV-OMua4WDsz0PUztqr1uEH5 2019-12-09T10:08:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.884+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.884+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.884+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 2019-12-09T10:08:29.985 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.992+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7]) start _multipart_[] num_entries 1001 2019-12-09T10:08:29.985 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:29.992+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:30.075 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.082+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.145+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x55c6b93baac0 s->prefetch_data=0 2019-12-09T10:08:30.139 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.145+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-12-09T10:08:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T10:08:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:30.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:30.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-12-09T10:08:30.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.170+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-12-09T10:08:30.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.182+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552266 cookie 94311998969904 notifier 4493 bl.length()=189 2019-12-09T10:08:30.591 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.598+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7 from cache 2019-12-09T10:08:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.598+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=246 2019-12-09T10:08:30.610 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:30.617+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552260 cookie 94311998968000 notifier 4493 bl.length()=246 2019-12-09T10:08:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.694+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.694+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-12-09T10:08:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.712+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-12-09T10:08:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.712+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T10:08:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.712+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.712+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.712+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.712+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.728+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=237 2019-12-09T10:08:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.728+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-12-09T10:08:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.728+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T10:08:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.728+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 state=0x55c6b93baac0 s->prefetch_data=0 2019-12-09T10:08:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.728+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 : miss 2019-12-09T10:08:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 info.flags=0x16 2019-12-09T10:08:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 to cache LRU end 2019-12-09T10:08:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.752+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:31.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.773+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=329 2019-12-09T10:08:31.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.773+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 info.flags=0x13 2019-12-09T10:08:31.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.773+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 to cache LRU end 2019-12-09T10:08:31.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.773+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:31.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.773+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:31.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.773+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.788+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9]) start [] num_entries 1100 2019-12-09T10:08:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T10:08:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T10:08:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T10:08:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T10:08:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T10:08:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T10:08:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T10:08:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T10:08:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T10:08:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T10:08:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T10:08:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T10:08:31.810 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.815+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55c6b93ba098 s->prefetch_data=0 2019-12-09T10:08:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.836+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.836+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.224 2019-12-09T10:08:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.836+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55c6b93ba098 s->prefetch_data=0 2019-12-09T10:08:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.836+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93ba098 2019-12-09T10:08:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:31.836+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T10:08:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.015+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T10:08:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.016+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.224 2019-12-09T10:08:32.102 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.109+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.127+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:32.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.127+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.225 2019-12-09T10:08:32.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.127+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.127+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:32.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.127+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:32.269 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T10:08:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:32.272 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T10:08:32.272 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.276+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.225 2019-12-09T10:08:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.347+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.359+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:32.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.359+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _2OF0GHFmK9NWT-97S5xrE1Vyi8MXEAB 2019-12-09T10:08:32.353 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.359+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.353 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.359+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:32.353 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.359+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:32.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2019-12-09T10:08:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:32.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T10:08:32.447 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.452+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _2OF0GHFmK9NWT-97S5xrE1Vyi8MXEAB 2019-12-09T10:08:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.521+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.537+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.537+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _J2cOMSBceoxIcY6jDqW5WBIeQDe2LhF 2019-12-09T10:08:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.537+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.537+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.537+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:32.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.661+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:32.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.661+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2019-12-09T10:08:32.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.661+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:32.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.661+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T10:08:32.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.661+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: _J2cOMSBceoxIcY6jDqW5WBIeQDe2LhF 2019-12-09T10:08:32.764 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.771+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.779+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.779+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.233 2019-12-09T10:08:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.779+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.779+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.779+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T10:08:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T10:08:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.909+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.233 2019-12-09T10:08:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.954+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.963+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.963+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _PVyRD5UM-UP6YBldW0L1CO_v-GOeZ4l 2019-12-09T10:08:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.963+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.964+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:32.964+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2019-12-09T10:08:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T10:08:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.071+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: _PVyRD5UM-UP6YBldW0L1CO_v-GOeZ4l 2019-12-09T10:08:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.157+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.159 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.166+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:33.159 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.166+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.235 2019-12-09T10:08:33.159 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.166+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.159 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.166+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.166+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T10:08:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T10:08:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.258+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.235 2019-12-09T10:08:33.342 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.349+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.366+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.366+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _O8GsotdqQnKz9gygEiBuHdFhmsAgZuc 2019-12-09T10:08:33.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.366+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.366+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:33.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.366+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.530+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.530+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2019-12-09T10:08:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.530+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.530+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T10:08:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.530+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: _O8GsotdqQnKz9gygEiBuHdFhmsAgZuc 2019-12-09T10:08:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.631+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.644+0000 7f5294b78ec0 10 manifest: total_size = 541681 2019-12-09T10:08:33.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.644+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.242 2019-12-09T10:08:33.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.644+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.644+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:33.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.644+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:33.786 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=246247 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4436455 part_ofs=246247 rule->part_size=246247 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2019-12-09T10:08:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4682702 part_ofs=492494 rule->part_size=49187 2019-12-09T10:08:33.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2019-12-09T10:08:33.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.793+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.242 2019-12-09T10:08:33.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.933+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.951+0000 7f5294b78ec0 10 manifest: total_size = 541681 2019-12-09T10:08:33.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.951+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _68KxTlJOIdEVN9J6oQflCha3W0ph-g5 2019-12-09T10:08:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.951+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.951+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:33.951+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.100+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.100+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=0 rule->part_size=0 2019-12-09T10:08:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.100+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _68KxTlJOIdEVN9J6oQflCha3W0ph-g5 2019-12-09T10:08:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.172+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:34.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.181+0000 7f5294b78ec0 10 manifest: total_size = 541681 2019-12-09T10:08:34.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.181+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to _rH5av0TLo7BZ13AdlcnFhcXmAUSF6QT 2019-12-09T10:08:34.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.181+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:34.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.181+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:34.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.181+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:34.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.285+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:34.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.285+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=0 rule->part_size=0 2019-12-09T10:08:34.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.286+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: _rH5av0TLo7BZ13AdlcnFhcXmAUSF6QT 2019-12-09T10:08:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.344+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.356+0000 7f5294b78ec0 10 manifest: total_size = 541681 2019-12-09T10:08:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.356+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.245 2019-12-09T10:08:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.356+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55c6b93be928 s->prefetch_data=0 2019-12-09T10:08:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.356+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93be928 2019-12-09T10:08:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.356+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 2019-12-09T10:08:34.475 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T10:08:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=246247 2019-12-09T10:08:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-09T10:08:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T10:08:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4436455 part_ofs=246247 rule->part_size=246247 2019-12-09T10:08:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2019-12-09T10:08:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2019-12-09T10:08:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4682702 part_ofs=492494 rule->part_size=49187 2019-12-09T10:08:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2019-12-09T10:08:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.482+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.245 2019-12-09T10:08:34.529 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.536+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9]) start _multipart_[] num_entries 1001 2019-12-09T10:08:34.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.536+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:34.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.547+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.594+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.594+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:34.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-12-09T10:08:34.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T10:08:34.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-12-09T10:08:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.612+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-12-09T10:08:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.618+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552258 cookie 94311998955056 notifier 4493 bl.length()=199 2019-12-09T10:08:34.734 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.741+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9 from cache 2019-12-09T10:08:34.734 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.741+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=256 2019-12-09T10:08:34.754 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:34.761+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552267 cookie 94311998969904 notifier 4493 bl.length()=256 2019-12-09T10:08:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.071+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.071+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-12-09T10:08:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.090+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-12-09T10:08:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.090+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T10:08:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.090+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.090+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.090+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.090+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:35.100 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.107+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=232 2019-12-09T10:08:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.107+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-12-09T10:08:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.107+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T10:08:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.107+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.107+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 : miss 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 info.flags=0x16 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 to cache LRU end 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:35.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:35.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:35.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.119+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:35.139 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.146+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=324 2019-12-09T10:08:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.146+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 info.flags=0x13 2019-12-09T10:08:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.146+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 to cache LRU end 2019-12-09T10:08:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.146+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.146+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.146+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 2019-12-09T10:08:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.170+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8]) start [] num_entries 1100 2019-12-09T10:08:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.204+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T10:08:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.204+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T10:08:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.204+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T10:08:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.204+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.227+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.227+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.196 2019-12-09T10:08:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.227+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.227+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.227+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 2019-12-09T10:08:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T10:08:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T10:08:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.355+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.196 2019-12-09T10:08:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.439+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.452+0000 7f5294b78ec0 10 manifest: total_size = 11534336 2019-12-09T10:08:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.452+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.203 2019-12-09T10:08:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.452+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.452+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.452+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 2019-12-09T10:08:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T10:08:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T10:08:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T10:08:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T10:08:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T10:08:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T10:08:35.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.570+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.203 2019-12-09T10:08:35.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.639+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.656+0000 7f5294b78ec0 10 manifest: total_size = 1140 2019-12-09T10:08:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.656+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.210 2019-12-09T10:08:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.656+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.656+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.656+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 2019-12-09T10:08:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=515 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4190723 part_ofs=515 rule->part_size=515 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=110 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=110 rules.size()=2 2019-12-09T10:08:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): stripe_ofs=4191238 part_ofs=1030 rule->part_size=110 2019-12-09T10:08:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=1140 rule->part_size=110 2019-12-09T10:08:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.775+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.210 2019-12-09T10:08:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.846+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8]) start _multipart_[] num_entries 1001 2019-12-09T10:08:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.846+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:35.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.863+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.919+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.919+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-12-09T10:08:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T10:08:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-12-09T10:08:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.933+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-12-09T10:08:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:35.950+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552261 cookie 94311998959280 notifier 4493 bl.length()=194 2019-12-09T10:08:36.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.466+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8 from cache 2019-12-09T10:08:36.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.466+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=251 2019-12-09T10:08:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.481+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552268 cookie 94311998969904 notifier 4493 bl.length()=251 2019-12-09T10:08:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.838+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.838+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-12-09T10:08:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.849+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-12-09T10:08:36.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.849+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T10:08:36.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.849+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:36.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.849+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:36.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.849+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:36.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.849+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:36.854 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.861+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=222 2019-12-09T10:08:36.854 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.861+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-12-09T10:08:36.854 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.861+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T10:08:36.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.861+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:36.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.861+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 : miss 2019-12-09T10:08:36.883 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 info.flags=0x16 2019-12-09T10:08:36.883 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 to cache LRU end 2019-12-09T10:08:36.883 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:36.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:36.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:36.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:36.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:36.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:36.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.890+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:36.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.921+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=314 2019-12-09T10:08:36.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.921+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 info.flags=0x13 2019-12-09T10:08:36.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.921+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 to cache LRU end 2019-12-09T10:08:36.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.921+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:36.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.921+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:36.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.921+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 2019-12-09T10:08:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.943+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6]) start [] num_entries 1100 2019-12-09T10:08:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.974+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T10:08:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.974+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T10:08:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.974+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T10:08:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.974+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:36.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.995+0000 7f5294b78ec0 10 manifest: total_size = 10485760 2019-12-09T10:08:36.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.995+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.84 2019-12-09T10:08:36.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.995+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:36.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.995+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:36.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:36.995+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 2019-12-09T10:08:37.152 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2019-12-09T10:08:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T10:08:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.159+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.84 2019-12-09T10:08:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.219+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:37.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.229+0000 7f5294b78ec0 10 manifest: total_size = 10485760 2019-12-09T10:08:37.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.229+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.85 2019-12-09T10:08:37.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.229+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:37.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.229+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:37.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.229+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 2019-12-09T10:08:37.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:37.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2019-12-09T10:08:37.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:37.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2019-12-09T10:08:37.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T10:08:37.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T10:08:37.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.364+0000 7f5294b78ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.85 2019-12-09T10:08:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.418+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.427+0000 7f5294b78ec0 10 manifest: total_size = 648332 2019-12-09T10:08:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.427+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.86 2019-12-09T10:08:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.427+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.427+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.427+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 2019-12-09T10:08:37.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.557+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6]) start _multipart_[] num_entries 1001 2019-12-09T10:08:37.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.557+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:37.618 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.625+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:37.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.692+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:37.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.692+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:37.709 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-12-09T10:08:37.709 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T10:08:37.709 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:37.710 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:37.710 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:37.710 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-12-09T10:08:37.710 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.716+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-12-09T10:08:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.733+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552269 cookie 94311998969904 notifier 4493 bl.length()=184 2019-12-09T10:08:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:37.763+0000 7f52527dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:08:38.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.119+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6 from cache 2019-12-09T10:08:38.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.119+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=241 2019-12-09T10:08:38.127 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.133+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552259 cookie 94311998971296 notifier 4493 bl.length()=241 2019-12-09T10:08:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.469+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.469+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-12-09T10:08:38.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.480+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-12-09T10:08:38.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.480+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T10:08:38.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.480+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:38.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.480+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:38.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.480+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:38.474 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.480+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:38.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.497+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T10:08:38.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.497+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-12-09T10:08:38.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.497+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T10:08:38.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.497+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:38.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.497+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 : miss 2019-12-09T10:08:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 info.flags=0x16 2019-12-09T10:08:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 to cache LRU end 2019-12-09T10:08:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:38.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:38.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:38.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.516+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:38.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.546+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T10:08:38.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.546+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 info.flags=0x13 2019-12-09T10:08:38.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.546+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 to cache LRU end 2019-12-09T10:08:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.546+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.546+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.546+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 2019-12-09T10:08:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.571+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10]) start [] num_entries 1100 2019-12-09T10:08:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.586+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T10:08:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.586+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T10:08:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.586+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T10:08:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.586+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T10:08:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.586+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.602+0000 7f5294b78ec0 10 manifest: total_size = 1048576 2019-12-09T10:08:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.602+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.305 2019-12-09T10:08:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.602+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.602+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:38.596 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.602+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 2019-12-09T10:08:38.749 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.755+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.768+0000 7f5294b78ec0 10 manifest: total_size = 110 2019-12-09T10:08:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.768+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.307 2019-12-09T10:08:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.768+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.768+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.768+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 2019-12-09T10:08:38.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.951+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.964+0000 7f5294b78ec0 10 manifest: total_size = 49187 2019-12-09T10:08:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.964+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.306 2019-12-09T10:08:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.964+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.964+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:38.964+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 2019-12-09T10:08:39.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.160+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.176+0000 7f5294b78ec0 10 manifest: total_size = 49187 2019-12-09T10:08:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.176+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.304 2019-12-09T10:08:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.176+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.176+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.176+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 2019-12-09T10:08:39.338 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.344+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10]) start _multipart_[] num_entries 1001 2019-12-09T10:08:39.338 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.344+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:39.439 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.446+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:39.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.518+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:39.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.518+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-12-09T10:08:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T10:08:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-12-09T10:08:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.533+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T10:08:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.543+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552262 cookie 94311998968000 notifier 4493 bl.length()=181 2019-12-09T10:08:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.656+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.10 from cache 2019-12-09T10:08:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.656+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=239 2019-12-09T10:08:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.668+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552263 cookie 94311998968000 notifier 4493 bl.length()=239 2019-12-09T10:08:39.953 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.961+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:39.954 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.961+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-12-09T10:08:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.980+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-12-09T10:08:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.980+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T10:08:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.980+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.980+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.980+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:39.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:39.980+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.004+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T10:08:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.004+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-12-09T10:08:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.004+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T10:08:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.004+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.004+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 : miss 2019-12-09T10:08:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 info.flags=0x16 2019-12-09T10:08:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 to cache LRU end 2019-12-09T10:08:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.019+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.043+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T10:08:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.043+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 info.flags=0x13 2019-12-09T10:08:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.043+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 to cache LRU end 2019-12-09T10:08:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.043+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.043+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.043+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 2019-12-09T10:08:40.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.056+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11]) start [] num_entries 1100 2019-12-09T10:08:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.076+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T10:08:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.076+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T10:08:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.076+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T10:08:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.076+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T10:08:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.076+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.077 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.085+0000 7f5294b78ec0 10 manifest: total_size = 2078 2019-12-09T10:08:40.077 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.085+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.312 2019-12-09T10:08:40.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.085+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.085+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:40.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.085+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 2019-12-09T10:08:40.210 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.217+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.229+0000 7f5294b78ec0 10 manifest: total_size = 62 2019-12-09T10:08:40.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.229+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.314 2019-12-09T10:08:40.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.229+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.229+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:40.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.229+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 2019-12-09T10:08:40.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.367+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.375+0000 7f5294b78ec0 10 manifest: total_size = 527 2019-12-09T10:08:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.375+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.313 2019-12-09T10:08:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.375+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.375+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.375+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 2019-12-09T10:08:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.596+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.615+0000 7f5294b78ec0 10 manifest: total_size = 24595 2019-12-09T10:08:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.615+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.311 2019-12-09T10:08:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.615+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55c6b93bce08 s->prefetch_data=0 2019-12-09T10:08:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.615+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bce08 2019-12-09T10:08:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.615+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 2019-12-09T10:08:40.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.737+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11]) start _multipart_[] num_entries 1001 2019-12-09T10:08:40.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.737+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.837+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.934+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x55c6b93beda0 s->prefetch_data=0 2019-12-09T10:08:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.934+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:40.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-12-09T10:08:40.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T10:08:40.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:40.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:40.946 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:40.946 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-12-09T10:08:40.946 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.953+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T10:08:40.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:40.967+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552264 cookie 94311998968000 notifier 4493 bl.length()=181 2019-12-09T10:08:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.213+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.11 from cache 2019-12-09T10:08:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.213+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=239 2019-12-09T10:08:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.237+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552265 cookie 94311998968000 notifier 4493 bl.length()=239 2019-12-09T10:08:41.793 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.801+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:41.793 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.801+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-12-09T10:08:41.812 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.820+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-12-09T10:08:41.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.820+0000 7f5294b78ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T10:08:41.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.820+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:41.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.820+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:41.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.820+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-12-09T10:08:41.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.820+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.836+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T10:08:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.836+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-12-09T10:08:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.836+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T10:08:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.836+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b605210 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 state=0x55c6b93c0020 s->prefetch_data=0 2019-12-09T10:08:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.836+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 : miss 2019-12-09T10:08:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 info.flags=0x16 2019-12-09T10:08:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 to cache LRU end 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 20 Read xattr: user.rgw.acl 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 20 Read xattr: user.rgw.idtag 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 : type miss (requested=0x13, cached=0x16) 2019-12-09T10:08:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.864+0000 7f5294b78ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:41.884 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.892+0000 7f5294b78ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T10:08:41.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.892+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 info.flags=0x13 2019-12-09T10:08:41.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.892+0000 7f5294b78ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 to cache LRU end 2019-12-09T10:08:41.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.892+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:41.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.892+0000 7f5294b78ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T10:08:41.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.892+0000 7f5294b78ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 2019-12-09T10:08:41.910 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.918+0000 7f5294b78ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12]) start [] num_entries 1100 2019-12-09T10:08:41.929 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.937+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T10:08:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.937+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T10:08:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.937+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T10:08:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.937+0000 7f5294b78ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T10:08:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.937+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.951+0000 7f5294b78ec0 10 manifest: total_size = 0 2019-12-09T10:08:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.951+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.319 2019-12-09T10:08:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.951+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.951+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bd178 2019-12-09T10:08:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:41.951+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 2019-12-09T10:08:42.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.093+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:42.108 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.116+0000 7f5294b78ec0 10 manifest: total_size = 0 2019-12-09T10:08:42.108 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.116+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.321 2019-12-09T10:08:42.109 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.116+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:42.109 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.116+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bd178 2019-12-09T10:08:42.109 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.116+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 2019-12-09T10:08:42.234 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.242+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.265+0000 7f5294b78ec0 10 manifest: total_size = 0 2019-12-09T10:08:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.265+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.320 2019-12-09T10:08:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.265+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.265+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bd178 2019-12-09T10:08:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.265+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 2019-12-09T10:08:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.396+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.404+0000 7f5294b78ec0 10 manifest: total_size = 0 2019-12-09T10:08:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.405+0000 7f5294b78ec0 20 get_obj_state: setting s->obj_tag to 478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.318 2019-12-09T10:08:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.405+0000 7f5294b78ec0 20 get_obj_state: rctx=0x7ffe6b605230 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55c6b93bd178 s->prefetch_data=0 2019-12-09T10:08:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.405+0000 7f5294b78ec0 20 prepare_atomic_modification: state is not atomic. state=0x55c6b93bd178 2019-12-09T10:08:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.405+0000 7f5294b78ec0 20 bucket index object: default.rgw.buckets.index:.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 2019-12-09T10:08:42.537 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.545+0000 7f5294b78ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12]) start _multipart_[] num_entries 1001 2019-12-09T10:08:42.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.545+0000 7f5294b78ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T10:08:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.607+0000 7f5294b78ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12])"; objs.size()=0; is_truncated=0 2019-12-09T10:08:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.697+0000 7f5294b78ec0 20 get_system_obj_state: rctx=0x7ffe6b604bc8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55c6b93b9e40 s->prefetch_data=0 2019-12-09T10:08:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.697+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-12-09T10:08:42.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-12-09T10:08:42.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T10:08:42.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:08:42.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:08:42.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-12-09T10:08:42.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-12-09T10:08:42.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.712+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T10:08:42.724 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.732+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552266 cookie 94311998968000 notifier 4493 bl.length()=181 2019-12-09T10:08:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.968+0000 7f5294b78ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.12 from cache 2019-12-09T10:08:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.968+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=239 2019-12-09T10:08:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:42.986+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552267 cookie 94311998968000 notifier 4493 bl.length()=239 2019-12-09T10:08:43.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:43.542+0000 7f5294b78ec0 10 removing key index: SIPGHYIHMETDGLIVBURF 2019-12-09T10:08:43.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:43.542+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=170 2019-12-09T10:08:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:43.564+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552270 cookie 94311998969904 notifier 4493 bl.length()=170 2019-12-09T10:08:43.845 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:43.853+0000 7f5294b78ec0 10 removing email index: ragweed.client.0+test@test.test 2019-12-09T10:08:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:43.853+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-12-09T10:08:43.869 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:43.877+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552271 cookie 94311998969904 notifier 4493 bl.length()=182 2019-12-09T10:08:44.404 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.412+0000 7f5294b78ec0 10 removing user buckets index 2019-12-09T10:08:44.405 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.412+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-12-09T10:08:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.439+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552268 cookie 94311998959280 notifier 4493 bl.length()=173 2019-12-09T10:08:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.715+0000 7f5294b78ec0 10 removing user index: ragweed.client.0 2019-12-09T10:08:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.715+0000 7f5294b78ec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-12-09T10:08:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.715+0000 7f5294b78ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-12-09T10:08:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.775+0000 7f5262ffd700 10 RGWWatcher::handle_notify() notify_id 584115552272 cookie 94311998957296 notifier 4493 bl.length()=165 2019-12-09T10:08:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.984+0000 7f5294b78ec0 20 remove_watcher() i=0 2019-12-09T10:08:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:44.984+0000 7f5294b78ec0 2 removed watcher, disabling cache 2019-12-09T10:08:45.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.026+0000 7f5294b78ec0 20 remove_watcher() i=1 2019-12-09T10:08:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.063+0000 7f5294b78ec0 20 remove_watcher() i=2 2019-12-09T10:08:45.104 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.111+0000 7f5294b78ec0 20 remove_watcher() i=3 2019-12-09T10:08:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.158+0000 7f5294b78ec0 20 remove_watcher() i=4 2019-12-09T10:08:45.194 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.202+0000 7f5294b78ec0 20 remove_watcher() i=5 2019-12-09T10:08:45.267 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.261+0000 7f5294b78ec0 20 remove_watcher() i=6 2019-12-09T10:08:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:45.307+0000 7f5294b78ec0 20 remove_watcher() i=7 2019-12-09T10:08:45.307 INFO:tasks.ragweed:Removing ragweed... 2019-12-09T10:08:45.308 INFO:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/ragweed 2019-12-09T10:08:45.400 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-12-09T10:08:45.448 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-12-09T10:08:45.449 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-12-09T10:08:45.449 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-12-09T10:08:45.449 INFO:teuthology.orchestra.run.smithi059:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-12-09T10:08:45.513 INFO:teuthology.orchestra.run.smithi059.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-12-09T10:08:49.361 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-12-09T10:08:49.361 INFO:teuthology.orchestra.run.smithi059:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-12-09T10:08:51.977 INFO:teuthology.orchestra.run.smithi059.stdout: Gradle will be installed under /opt/gradle 2019-12-09T10:08:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:--2019-12-09 10:08:52-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-12-09T10:08:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.190.9, 104.18.191.9, 2606:4700::6812:bf09, ... 2019-12-09T10:08:52.061 INFO:teuthology.orchestra.run.smithi059.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.190.9|:443... connected. 2019-12-09T10:08:52.093 INFO:teuthology.orchestra.run.smithi059.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-12-09T10:08:52.094 INFO:teuthology.orchestra.run.smithi059.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-12-09T10:08:52.094 INFO:teuthology.orchestra.run.smithi059.stderr:--2019-12-09 10:08:52-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-12-09T10:08:52.094 INFO:teuthology.orchestra.run.smithi059.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.160.20, 104.17.159.20, 2606:4700::6811:9f14, ... 2019-12-09T10:08:52.102 INFO:teuthology.orchestra.run.smithi059.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.160.20|:443... connected. 2019-12-09T10:08:52.150 INFO:teuthology.orchestra.run.smithi059.stderr:HTTP request sent, awaiting response... 200 OK 2019-12-09T10:08:52.150 INFO:teuthology.orchestra.run.smithi059.stderr:Length: 75323957 (72M) [application/zip] 2019-12-09T10:08:52.150 INFO:teuthology.orchestra.run.smithi059.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-12-09T10:08:52.150 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T10:08:52.158 INFO:teuthology.orchestra.run.smithi059.stderr: 0K .......... .......... .......... .......... .......... 0% 6.75M 11s 2019-12-09T10:08:52.158 INFO:teuthology.orchestra.run.smithi059.stderr: 50K .......... .......... .......... .......... .......... 0% 94.1M 6s 2019-12-09T10:08:52.163 INFO:teuthology.orchestra.run.smithi059.stderr: 100K .......... .......... .......... .......... .......... 0% 7.25M 7s 2019-12-09T10:08:52.167 INFO:teuthology.orchestra.run.smithi059.stderr: 150K .......... .......... .......... .......... .......... 0% 90.6M 6s 2019-12-09T10:08:52.168 INFO:teuthology.orchestra.run.smithi059.stderr: 200K .......... .......... .......... .......... .......... 0% 126M 5s 2019-12-09T10:08:52.168 INFO:teuthology.orchestra.run.smithi059.stderr: 250K .......... .......... .......... .......... .......... 0% 153M 4s 2019-12-09T10:08:52.175 INFO:teuthology.orchestra.run.smithi059.stderr: 300K .......... .......... .......... .......... .......... 0% 7.96M 5s 2019-12-09T10:08:52.175 INFO:teuthology.orchestra.run.smithi059.stderr: 350K .......... .......... .......... .......... .......... 0% 134M 4s 2019-12-09T10:08:52.175 INFO:teuthology.orchestra.run.smithi059.stderr: 400K .......... .......... .......... .......... .......... 0% 174M 4s 2019-12-09T10:08:52.175 INFO:teuthology.orchestra.run.smithi059.stderr: 450K .......... .......... .......... .......... .......... 0% 126M 3s 2019-12-09T10:08:52.175 INFO:teuthology.orchestra.run.smithi059.stderr: 500K .......... .......... .......... .......... .......... 0% 138M 3s 2019-12-09T10:08:52.175 INFO:teuthology.orchestra.run.smithi059.stderr: 550K .......... .......... .......... .......... .......... 0% 121M 3s 2019-12-09T10:08:52.180 INFO:teuthology.orchestra.run.smithi059.stderr: 600K .......... .......... .......... .......... .......... 0% 8.31M 3s 2019-12-09T10:08:52.181 INFO:teuthology.orchestra.run.smithi059.stderr: 650K .......... .......... .......... .......... .......... 0% 156M 3s 2019-12-09T10:08:52.181 INFO:teuthology.orchestra.run.smithi059.stderr: 700K .......... .......... .......... .......... .......... 1% 117M 3s 2019-12-09T10:08:52.181 INFO:teuthology.orchestra.run.smithi059.stderr: 750K .......... .......... .......... .......... .......... 1% 114M 3s 2019-12-09T10:08:52.188 INFO:teuthology.orchestra.run.smithi059.stderr: 800K .......... .......... .......... .......... .......... 1% 141M 3s 2019-12-09T10:08:52.189 INFO:teuthology.orchestra.run.smithi059.stderr: 850K .......... .......... .......... .......... .......... 1% 128M 3s 2019-12-09T10:08:52.189 INFO:teuthology.orchestra.run.smithi059.stderr: 900K .......... .......... .......... .......... .......... 1% 125M 2s 2019-12-09T10:08:52.189 INFO:teuthology.orchestra.run.smithi059.stderr: 950K .......... .......... .......... .......... .......... 1% 109M 2s 2019-12-09T10:08:52.189 INFO:teuthology.orchestra.run.smithi059.stderr: 1000K .......... .......... .......... .......... .......... 1% 111M 2s 2019-12-09T10:08:52.189 INFO:teuthology.orchestra.run.smithi059.stderr: 1050K .......... .......... .......... .......... .......... 1% 119M 2s 2019-12-09T10:08:52.189 INFO:teuthology.orchestra.run.smithi059.stderr: 1100K .......... .......... .......... .......... .......... 1% 134M 2s 2019-12-09T10:08:52.190 INFO:teuthology.orchestra.run.smithi059.stderr: 1150K .......... .......... .......... .......... .......... 1% 109M 2s 2019-12-09T10:08:53.258 INFO:teuthology.orchestra.run.smithi059.stderr: 1200K .......... .......... .......... .......... .......... 1% 137M 2s 2019-12-09T10:08:53.258 INFO:teuthology.orchestra.run.smithi059.stderr: 1250K .......... .......... .......... .......... .......... 1% 154M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1300K .......... .......... .......... .......... .......... 1% 153M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1350K .......... .......... .......... .......... .......... 1% 24.5M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1400K .......... .......... .......... .......... .......... 1% 171M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1450K .......... .......... .......... .......... .......... 2% 129M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1500K .......... .......... .......... .......... .......... 2% 147M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1550K .......... .......... .......... .......... .......... 2% 96.8M 2s 2019-12-09T10:08:53.259 INFO:teuthology.orchestra.run.smithi059.stderr: 1600K .......... .......... .......... .......... .......... 2% 113M 2s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1650K .......... .......... .......... .......... .......... 2% 129M 2s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1700K .......... .......... .......... .......... .......... 2% 190M 2s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1750K .......... .......... .......... .......... .......... 2% 130M 2s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1800K .......... .......... .......... .......... .......... 2% 264M 2s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1850K .......... .......... .......... .......... .......... 2% 117M 2s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1900K .......... .......... .......... .......... .......... 2% 157M 1s 2019-12-09T10:08:53.260 INFO:teuthology.orchestra.run.smithi059.stderr: 1950K .......... .......... .......... .......... .......... 2% 159M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2000K .......... .......... .......... .......... .......... 2% 130M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2050K .......... .......... .......... .......... .......... 2% 186M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2100K .......... .......... .......... .......... .......... 2% 118M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2150K .......... .......... .......... .......... .......... 2% 159M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2200K .......... .......... .......... .......... .......... 3% 27.6M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2250K .......... .......... .......... .......... .......... 3% 170M 1s 2019-12-09T10:08:53.261 INFO:teuthology.orchestra.run.smithi059.stderr: 2300K .......... .......... .......... .......... .......... 3% 164M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2350K .......... .......... .......... .......... .......... 3% 187M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2400K .......... .......... .......... .......... .......... 3% 190M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2450K .......... .......... .......... .......... .......... 3% 215M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2500K .......... .......... .......... .......... .......... 3% 178M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2550K .......... .......... .......... .......... .......... 3% 191M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2600K .......... .......... .......... .......... .......... 3% 211M 1s 2019-12-09T10:08:53.262 INFO:teuthology.orchestra.run.smithi059.stderr: 2650K .......... .......... .......... .......... .......... 3% 212M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 2700K .......... .......... .......... .......... .......... 3% 220M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 2750K .......... .......... .......... .......... .......... 3% 190M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 2800K .......... .......... .......... .......... .......... 3% 217M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 2850K .......... .......... .......... .......... .......... 3% 229M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 2900K .......... .......... .......... .......... .......... 4% 221M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 2950K .......... .......... .......... .......... .......... 4% 194M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 3000K .......... .......... .......... .......... .......... 4% 204M 1s 2019-12-09T10:08:53.263 INFO:teuthology.orchestra.run.smithi059.stderr: 3050K .......... .......... .......... .......... .......... 4% 230M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3100K .......... .......... .......... .......... .......... 4% 219M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3150K .......... .......... .......... .......... .......... 4% 200M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3200K .......... .......... .......... .......... .......... 4% 221M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3250K .......... .......... .......... .......... .......... 4% 222M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3300K .......... .......... .......... .......... .......... 4% 215M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3350K .......... .......... .......... .......... .......... 4% 192M 1s 2019-12-09T10:08:53.264 INFO:teuthology.orchestra.run.smithi059.stderr: 3400K .......... .......... .......... .......... .......... 4% 206M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3450K .......... .......... .......... .......... .......... 4% 186M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3500K .......... .......... .......... .......... .......... 4% 211M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3550K .......... .......... .......... .......... .......... 4% 169M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3600K .......... .......... .......... .......... .......... 4% 186M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3650K .......... .......... .......... .......... .......... 5% 191M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3700K .......... .......... .......... .......... .......... 5% 198M 1s 2019-12-09T10:08:53.265 INFO:teuthology.orchestra.run.smithi059.stderr: 3750K .......... .......... .......... .......... .......... 5% 146M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 3800K .......... .......... .......... .......... .......... 5% 183M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 3850K .......... .......... .......... .......... .......... 5% 189M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 3900K .......... .......... .......... .......... .......... 5% 196M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 3950K .......... .......... .......... .......... .......... 5% 169M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 4000K .......... .......... .......... .......... .......... 5% 180M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 4050K .......... .......... .......... .......... .......... 5% 191M 1s 2019-12-09T10:08:53.266 INFO:teuthology.orchestra.run.smithi059.stderr: 4100K .......... .......... .......... .......... .......... 5% 173M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4150K .......... .......... .......... .......... .......... 5% 168M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4200K .......... .......... .......... .......... .......... 5% 179M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4250K .......... .......... .......... .......... .......... 5% 145M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4300K .......... .......... .......... .......... .......... 5% 167M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4350K .......... .......... .......... .......... .......... 5% 183M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4400K .......... .......... .......... .......... .......... 6% 182M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4450K .......... .......... .......... .......... .......... 6% 185M 1s 2019-12-09T10:08:53.267 INFO:teuthology.orchestra.run.smithi059.stderr: 4500K .......... .......... .......... .......... .......... 6% 224M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4550K .......... .......... .......... .......... .......... 6% 179M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4600K .......... .......... .......... .......... .......... 6% 199M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4650K .......... .......... .......... .......... .......... 6% 200M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4700K .......... .......... .......... .......... .......... 6% 219M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4750K .......... .......... .......... .......... .......... 6% 193M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4800K .......... .......... .......... .......... .......... 6% 198M 1s 2019-12-09T10:08:53.268 INFO:teuthology.orchestra.run.smithi059.stderr: 4850K .......... .......... .......... .......... .......... 6% 167M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 4900K .......... .......... .......... .......... .......... 6% 227M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 4950K .......... .......... .......... .......... .......... 6% 224M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 5000K .......... .......... .......... .......... .......... 6% 208M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 5050K .......... .......... .......... .......... .......... 6% 210M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 5100K .......... .......... .......... .......... .......... 7% 272M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 5150K .......... .......... .......... .......... .......... 7% 203M 1s 2019-12-09T10:08:53.269 INFO:teuthology.orchestra.run.smithi059.stderr: 5200K .......... .......... .......... .......... .......... 7% 246M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5250K .......... .......... .......... .......... .......... 7% 122M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5300K .......... .......... .......... .......... .......... 7% 130M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5350K .......... .......... .......... .......... .......... 7% 148M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5400K .......... .......... .......... .......... .......... 7% 121M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5450K .......... .......... .......... .......... .......... 7% 138M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5500K .......... .......... .......... .......... .......... 7% 135M 1s 2019-12-09T10:08:53.270 INFO:teuthology.orchestra.run.smithi059.stderr: 5550K .......... .......... .......... .......... .......... 7% 121M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5600K .......... .......... .......... .......... .......... 7% 144M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5650K .......... .......... .......... .......... .......... 7% 150M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5700K .......... .......... .......... .......... .......... 7% 161M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5750K .......... .......... .......... .......... .......... 7% 149M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5800K .......... .......... .......... .......... .......... 7% 175M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5850K .......... .......... .......... .......... .......... 8% 149M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5900K .......... .......... .......... .......... .......... 8% 146M 1s 2019-12-09T10:08:53.271 INFO:teuthology.orchestra.run.smithi059.stderr: 5950K .......... .......... .......... .......... .......... 8% 114M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6000K .......... .......... .......... .......... .......... 8% 156M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6050K .......... .......... .......... .......... .......... 8% 153M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6100K .......... .......... .......... .......... .......... 8% 145M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6150K .......... .......... .......... .......... .......... 8% 137M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6200K .......... .......... .......... .......... .......... 8% 134M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6250K .......... .......... .......... .......... .......... 8% 144M 1s 2019-12-09T10:08:53.272 INFO:teuthology.orchestra.run.smithi059.stderr: 6300K .......... .......... .......... .......... .......... 8% 168M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6350K .......... .......... .......... .......... .......... 8% 141M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6400K .......... .......... .......... .......... .......... 8% 138M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6450K .......... .......... .......... .......... .......... 8% 186M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6500K .......... .......... .......... .......... .......... 8% 159M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6550K .......... .......... .......... .......... .......... 8% 149M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6600K .......... .......... .......... .......... .......... 9% 153M 1s 2019-12-09T10:08:53.273 INFO:teuthology.orchestra.run.smithi059.stderr: 6650K .......... .......... .......... .......... .......... 9% 174M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 6700K .......... .......... .......... .......... .......... 9% 185M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 6750K .......... .......... .......... .......... .......... 9% 134M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 6800K .......... .......... .......... .......... .......... 9% 167M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 6850K .......... .......... .......... .......... .......... 9% 160M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 6900K .......... .......... .......... .......... .......... 9% 160M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 6950K .......... .......... .......... .......... .......... 9% 178M 1s 2019-12-09T10:08:53.274 INFO:teuthology.orchestra.run.smithi059.stderr: 7000K .......... .......... .......... .......... .......... 9% 194M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7050K .......... .......... .......... .......... .......... 9% 163M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7100K .......... .......... .......... .......... .......... 9% 157M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7150K .......... .......... .......... .......... .......... 9% 160M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7200K .......... .......... .......... .......... .......... 9% 162M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7250K .......... .......... .......... .......... .......... 9% 161M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7300K .......... .......... .......... .......... .......... 9% 161M 1s 2019-12-09T10:08:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 7350K .......... .......... .......... .......... .......... 10% 139M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7400K .......... .......... .......... .......... .......... 10% 179M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7450K .......... .......... .......... .......... .......... 10% 154M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7500K .......... .......... .......... .......... .......... 10% 158M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7550K .......... .......... .......... .......... .......... 10% 136M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7600K .......... .......... .......... .......... .......... 10% 197M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7650K .......... .......... .......... .......... .......... 10% 149M 1s 2019-12-09T10:08:53.276 INFO:teuthology.orchestra.run.smithi059.stderr: 7700K .......... .......... .......... .......... .......... 10% 178M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 7750K .......... .......... .......... .......... .......... 10% 130M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 7800K .......... .......... .......... .......... .......... 10% 164M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 7850K .......... .......... .......... .......... .......... 10% 146M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 7900K .......... .......... .......... .......... .......... 10% 172M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 7950K .......... .......... .......... .......... .......... 10% 156M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 8000K .......... .......... .......... .......... .......... 10% 168M 1s 2019-12-09T10:08:53.277 INFO:teuthology.orchestra.run.smithi059.stderr: 8050K .......... .......... .......... .......... .......... 11% 148M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8100K .......... .......... .......... .......... .......... 11% 160M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8150K .......... .......... .......... .......... .......... 11% 184M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8200K .......... .......... .......... .......... .......... 11% 210M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8250K .......... .......... .......... .......... .......... 11% 150M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8300K .......... .......... .......... .......... .......... 11% 188M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8350K .......... .......... .......... .......... .......... 11% 136M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8400K .......... .......... .......... .......... .......... 11% 166M 1s 2019-12-09T10:08:53.278 INFO:teuthology.orchestra.run.smithi059.stderr: 8450K .......... .......... .......... .......... .......... 11% 179M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8500K .......... .......... .......... .......... .......... 11% 171M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8550K .......... .......... .......... .......... .......... 11% 148M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8600K .......... .......... .......... .......... .......... 11% 204M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8650K .......... .......... .......... .......... .......... 11% 168M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8700K .......... .......... .......... .......... .......... 11% 160M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8750K .......... .......... .......... .......... .......... 11% 138M 1s 2019-12-09T10:08:53.279 INFO:teuthology.orchestra.run.smithi059.stderr: 8800K .......... .......... .......... .......... .......... 12% 192M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 8850K .......... .......... .......... .......... .......... 12% 174M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 8900K .......... .......... .......... .......... .......... 12% 151M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 8950K .......... .......... .......... .......... .......... 12% 148M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 9000K .......... .......... .......... .......... .......... 12% 151M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 9050K .......... .......... .......... .......... .......... 12% 179M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 9100K .......... .......... .......... .......... .......... 12% 155M 1s 2019-12-09T10:08:53.280 INFO:teuthology.orchestra.run.smithi059.stderr: 9150K .......... .......... .......... .......... .......... 12% 146M 1s 2019-12-09T10:08:53.281 INFO:teuthology.orchestra.run.smithi059.stderr: 9200K .......... .......... .......... .......... .......... 12% 203M 1s 2019-12-09T10:08:53.281 INFO:teuthology.orchestra.run.smithi059.stderr: 9250K .......... .......... .......... .......... .......... 12% 209M 1s 2019-12-09T10:08:53.281 INFO:teuthology.orchestra.run.smithi059.stderr: 9300K .......... .......... .......... .......... .......... 12% 144M 1s 2019-12-09T10:08:53.281 INFO:teuthology.orchestra.run.smithi059.stderr: 9350K .......... .......... .......... .......... .......... 12% 154M 1s 2019-12-09T10:08:53.281 INFO:teuthology.orchestra.run.smithi059.stderr: 9400K .......... .......... .......... .......... .......... 12% 177M 1s 2019-12-09T10:08:53.281 INFO:teuthology.orchestra.run.smithi059.stderr: 9450K .......... .......... .......... .......... .......... 12% 187M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9500K .......... .......... .......... .......... .......... 12% 175M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9550K .......... .......... .......... .......... .......... 13% 141M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9600K .......... .......... .......... .......... .......... 13% 189M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9650K .......... .......... .......... .......... .......... 13% 212M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9700K .......... .......... .......... .......... .......... 13% 157M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9750K .......... .......... .......... .......... .......... 13% 172M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9800K .......... .......... .......... .......... .......... 13% 168M 1s 2019-12-09T10:08:53.282 INFO:teuthology.orchestra.run.smithi059.stderr: 9850K .......... .......... .......... .......... .......... 13% 198M 1s 2019-12-09T10:08:53.283 INFO:teuthology.orchestra.run.smithi059.stderr: 9900K .......... .......... .......... .......... .......... 13% 183M 1s 2019-12-09T10:08:53.283 INFO:teuthology.orchestra.run.smithi059.stderr: 9950K .......... .......... .......... .......... .......... 13% 153M 1s 2019-12-09T10:08:53.283 INFO:teuthology.orchestra.run.smithi059.stderr: 10000K .......... .......... .......... .......... .......... 13% 164M 1s 2019-12-09T10:08:53.283 INFO:teuthology.orchestra.run.smithi059.stderr: 10050K .......... .......... .......... .......... .......... 13% 216M 1s 2019-12-09T10:08:53.283 INFO:teuthology.orchestra.run.smithi059.stderr: 10100K .......... .......... .......... .......... .......... 13% 199M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10150K .......... .......... .......... .......... .......... 13% 151M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10200K .......... .......... .......... .......... .......... 13% 302M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10250K .......... .......... .......... .......... .......... 14% 253M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10300K .......... .......... .......... .......... .......... 14% 265M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10350K .......... .......... .......... .......... .......... 14% 212M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10400K .......... .......... .......... .......... .......... 14% 277M 1s 2019-12-09T10:08:53.284 INFO:teuthology.orchestra.run.smithi059.stderr: 10450K .......... .......... .......... .......... .......... 14% 248M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10500K .......... .......... .......... .......... .......... 14% 253M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10550K .......... .......... .......... .......... .......... 14% 276M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10600K .......... .......... .......... .......... .......... 14% 183M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10650K .......... .......... .......... .......... .......... 14% 204M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10700K .......... .......... .......... .......... .......... 14% 210M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10750K .......... .......... .......... .......... .......... 14% 117M 1s 2019-12-09T10:08:53.285 INFO:teuthology.orchestra.run.smithi059.stderr: 10800K .......... .......... .......... .......... .......... 14% 158M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 10850K .......... .......... .......... .......... .......... 14% 215M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 10900K .......... .......... .......... .......... .......... 14% 219M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 10950K .......... .......... .......... .......... .......... 14% 122M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 11000K .......... .......... .......... .......... .......... 15% 171M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 11050K .......... .......... .......... .......... .......... 15% 202M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 11100K .......... .......... .......... .......... .......... 15% 208M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 11150K .......... .......... .......... .......... .......... 15% 143M 1s 2019-12-09T10:08:53.286 INFO:teuthology.orchestra.run.smithi059.stderr: 11200K .......... .......... .......... .......... .......... 15% 158M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11250K .......... .......... .......... .......... .......... 15% 177M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11300K .......... .......... .......... .......... .......... 15% 186M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11350K .......... .......... .......... .......... .......... 15% 154M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11400K .......... .......... .......... .......... .......... 15% 189M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11450K .......... .......... .......... .......... .......... 15% 156M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11500K .......... .......... .......... .......... .......... 15% 190M 1s 2019-12-09T10:08:53.287 INFO:teuthology.orchestra.run.smithi059.stderr: 11550K .......... .......... .......... .......... .......... 15% 147M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11600K .......... .......... .......... .......... .......... 15% 157M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11650K .......... .......... .......... .......... .......... 15% 171M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11700K .......... .......... .......... .......... .......... 15% 190M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11750K .......... .......... .......... .......... .......... 16% 135M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11800K .......... .......... .......... .......... .......... 16% 172M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11850K .......... .......... .......... .......... .......... 16% 206M 1s 2019-12-09T10:08:53.288 INFO:teuthology.orchestra.run.smithi059.stderr: 11900K .......... .......... .......... .......... .......... 16% 181M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 11950K .......... .......... .......... .......... .......... 16% 171M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 12000K .......... .......... .......... .......... .......... 16% 170M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 12050K .......... .......... .......... .......... .......... 16% 160M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 12100K .......... .......... .......... .......... .......... 16% 215M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 12150K .......... .......... .......... .......... .......... 16% 182M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 12200K .......... .......... .......... .......... .......... 16% 215M 1s 2019-12-09T10:08:53.289 INFO:teuthology.orchestra.run.smithi059.stderr: 12250K .......... .......... .......... .......... .......... 16% 167M 1s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12300K .......... .......... .......... .......... .......... 16% 190M 1s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12350K .......... .......... .......... .......... .......... 16% 150M 1s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12400K .......... .......... .......... .......... .......... 16% 185M 0s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12450K .......... .......... .......... .......... .......... 16% 209M 0s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12500K .......... .......... .......... .......... .......... 17% 164M 0s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12550K .......... .......... .......... .......... .......... 17% 158M 0s 2019-12-09T10:08:53.290 INFO:teuthology.orchestra.run.smithi059.stderr: 12600K .......... .......... .......... .......... .......... 17% 210M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12650K .......... .......... .......... .......... .......... 17% 197M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12700K .......... .......... .......... .......... .......... 17% 201M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12750K .......... .......... .......... .......... .......... 17% 160M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12800K .......... .......... .......... .......... .......... 17% 175M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12850K .......... .......... .......... .......... .......... 17% 196M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12900K .......... .......... .......... .......... .......... 17% 158M 0s 2019-12-09T10:08:53.291 INFO:teuthology.orchestra.run.smithi059.stderr: 12950K .......... .......... .......... .......... .......... 17% 166M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13000K .......... .......... .......... .......... .......... 17% 172M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13050K .......... .......... .......... .......... .......... 17% 176M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13100K .......... .......... .......... .......... .......... 17% 151M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13150K .......... .......... .......... .......... .......... 17% 155M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13200K .......... .......... .......... .......... .......... 18% 220M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13250K .......... .......... .......... .......... .......... 18% 173M 0s 2019-12-09T10:08:53.292 INFO:teuthology.orchestra.run.smithi059.stderr: 13300K .......... .......... .......... .......... .......... 18% 173M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13350K .......... .......... .......... .......... .......... 18% 146M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13400K .......... .......... .......... .......... .......... 18% 169M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13450K .......... .......... .......... .......... .......... 18% 214M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13500K .......... .......... .......... .......... .......... 18% 194M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13550K .......... .......... .......... .......... .......... 18% 194M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13600K .......... .......... .......... .......... .......... 18% 144M 0s 2019-12-09T10:08:53.293 INFO:teuthology.orchestra.run.smithi059.stderr: 13650K .......... .......... .......... .......... .......... 18% 182M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 13700K .......... .......... .......... .......... .......... 18% 197M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 13750K .......... .......... .......... .......... .......... 18% 181M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 13800K .......... .......... .......... .......... .......... 18% 236M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 13850K .......... .......... .......... .......... .......... 18% 163M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 13900K .......... .......... .......... .......... .......... 18% 211M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 13950K .......... .......... .......... .......... .......... 19% 181M 0s 2019-12-09T10:08:53.294 INFO:teuthology.orchestra.run.smithi059.stderr: 14000K .......... .......... .......... .......... .......... 19% 167M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14050K .......... .......... .......... .......... .......... 19% 165M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14100K .......... .......... .......... .......... .......... 19% 231M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14150K .......... .......... .......... .......... .......... 19% 162M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14200K .......... .......... .......... .......... .......... 19% 179M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14250K .......... .......... .......... .......... .......... 19% 172M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14300K .......... .......... .......... .......... .......... 19% 229M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14350K .......... .......... .......... .......... .......... 19% 162M 0s 2019-12-09T10:08:53.295 INFO:teuthology.orchestra.run.smithi059.stderr: 14400K .......... .......... .......... .......... .......... 19% 165M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14450K .......... .......... .......... .......... .......... 19% 190M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14500K .......... .......... .......... .......... .......... 19% 171M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14550K .......... .......... .......... .......... .......... 19% 198M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14600K .......... .......... .......... .......... .......... 19% 168M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14650K .......... .......... .......... .......... .......... 19% 215M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14700K .......... .......... .......... .......... .......... 20% 152M 0s 2019-12-09T10:08:53.296 INFO:teuthology.orchestra.run.smithi059.stderr: 14750K .......... .......... .......... .......... .......... 20% 140M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 14800K .......... .......... .......... .......... .......... 20% 232M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 14850K .......... .......... .......... .......... .......... 20% 211M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 14900K .......... .......... .......... .......... .......... 20% 150M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 14950K .......... .......... .......... .......... .......... 20% 190M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 15000K .......... .......... .......... .......... .......... 20% 147M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 15050K .......... .......... .......... .......... .......... 20% 234M 0s 2019-12-09T10:08:53.297 INFO:teuthology.orchestra.run.smithi059.stderr: 15100K .......... .......... .......... .......... .......... 20% 185M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15150K .......... .......... .......... .......... .......... 20% 180M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15200K .......... .......... .......... .......... .......... 20% 168M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15250K .......... .......... .......... .......... .......... 20% 189M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15300K .......... .......... .......... .......... .......... 20% 213M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15350K .......... .......... .......... .......... .......... 20% 156M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15400K .......... .......... .......... .......... .......... 21% 201M 0s 2019-12-09T10:08:53.298 INFO:teuthology.orchestra.run.smithi059.stderr: 15450K .......... .......... .......... .......... .......... 21% 216M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15500K .......... .......... .......... .......... .......... 21% 173M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15550K .......... .......... .......... .......... .......... 21% 180M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15600K .......... .......... .......... .......... .......... 21% 180M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15650K .......... .......... .......... .......... .......... 21% 182M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15700K .......... .......... .......... .......... .......... 21% 218M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15750K .......... .......... .......... .......... .......... 21% 128M 0s 2019-12-09T10:08:53.299 INFO:teuthology.orchestra.run.smithi059.stderr: 15800K .......... .......... .......... .......... .......... 21% 210M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 15850K .......... .......... .......... .......... .......... 21% 180M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 15900K .......... .......... .......... .......... .......... 21% 229M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 15950K .......... .......... .......... .......... .......... 21% 151M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 16000K .......... .......... .......... .......... .......... 21% 197M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 16050K .......... .......... .......... .......... .......... 21% 194M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 16100K .......... .......... .......... .......... .......... 21% 188M 0s 2019-12-09T10:08:53.300 INFO:teuthology.orchestra.run.smithi059.stderr: 16150K .......... .......... .......... .......... .......... 22% 176M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16200K .......... .......... .......... .......... .......... 22% 232M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16250K .......... .......... .......... .......... .......... 22% 218M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16300K .......... .......... .......... .......... .......... 22% 142M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16350K .......... .......... .......... .......... .......... 22% 194M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16400K .......... .......... .......... .......... .......... 22% 146M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16450K .......... .......... .......... .......... .......... 22% 220M 0s 2019-12-09T10:08:53.301 INFO:teuthology.orchestra.run.smithi059.stderr: 16500K .......... .......... .......... .......... .......... 22% 172M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16550K .......... .......... .......... .......... .......... 22% 174M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16600K .......... .......... .......... .......... .......... 22% 173M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16650K .......... .......... .......... .......... .......... 22% 230M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16700K .......... .......... .......... .......... .......... 22% 211M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16750K .......... .......... .......... .......... .......... 22% 159M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16800K .......... .......... .......... .......... .......... 22% 182M 0s 2019-12-09T10:08:53.302 INFO:teuthology.orchestra.run.smithi059.stderr: 16850K .......... .......... .......... .......... .......... 22% 213M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 16900K .......... .......... .......... .......... .......... 23% 201M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 16950K .......... .......... .......... .......... .......... 23% 184M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 17000K .......... .......... .......... .......... .......... 23% 168M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 17050K .......... .......... .......... .......... .......... 23% 183M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 17100K .......... .......... .......... .......... .......... 23% 207M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 17150K .......... .......... .......... .......... .......... 23% 149M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 17200K .......... .......... .......... .......... .......... 23% 186M 0s 2019-12-09T10:08:53.303 INFO:teuthology.orchestra.run.smithi059.stderr: 17250K .......... .......... .......... .......... .......... 23% 193M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17300K .......... .......... .......... .......... .......... 23% 242M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17350K .......... .......... .......... .......... .......... 23% 169M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17400K .......... .......... .......... .......... .......... 23% 191M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17450K .......... .......... .......... .......... .......... 23% 242M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17500K .......... .......... .......... .......... .......... 23% 165M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17550K .......... .......... .......... .......... .......... 23% 175M 0s 2019-12-09T10:08:53.304 INFO:teuthology.orchestra.run.smithi059.stderr: 17600K .......... .......... .......... .......... .......... 23% 186M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17650K .......... .......... .......... .......... .......... 24% 197M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17700K .......... .......... .......... .......... .......... 24% 231M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17750K .......... .......... .......... .......... .......... 24% 165M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17800K .......... .......... .......... .......... .......... 24% 214M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17850K .......... .......... .......... .......... .......... 24% 200M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17900K .......... .......... .......... .......... .......... 24% 224M 0s 2019-12-09T10:08:53.305 INFO:teuthology.orchestra.run.smithi059.stderr: 17950K .......... .......... .......... .......... .......... 24% 154M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18000K .......... .......... .......... .......... .......... 24% 206M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18050K .......... .......... .......... .......... .......... 24% 210M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18100K .......... .......... .......... .......... .......... 24% 237M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18150K .......... .......... .......... .......... .......... 24% 180M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18200K .......... .......... .......... .......... .......... 24% 223M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18250K .......... .......... .......... .......... .......... 24% 195M 0s 2019-12-09T10:08:53.306 INFO:teuthology.orchestra.run.smithi059.stderr: 18300K .......... .......... .......... .......... .......... 24% 215M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18350K .......... .......... .......... .......... .......... 25% 179M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18400K .......... .......... .......... .......... .......... 25% 220M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18450K .......... .......... .......... .......... .......... 25% 160M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18500K .......... .......... .......... .......... .......... 25% 187M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18550K .......... .......... .......... .......... .......... 25% 156M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18600K .......... .......... .......... .......... .......... 25% 181M 0s 2019-12-09T10:08:53.307 INFO:teuthology.orchestra.run.smithi059.stderr: 18650K .......... .......... .......... .......... .......... 25% 202M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 18700K .......... .......... .......... .......... .......... 25% 204M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 18750K .......... .......... .......... .......... .......... 25% 183M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 18800K .......... .......... .......... .......... .......... 25% 191M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 18850K .......... .......... .......... .......... .......... 25% 195M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 18900K .......... .......... .......... .......... .......... 25% 229M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 18950K .......... .......... .......... .......... .......... 25% 182M 0s 2019-12-09T10:08:53.308 INFO:teuthology.orchestra.run.smithi059.stderr: 19000K .......... .......... .......... .......... .......... 25% 175M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19050K .......... .......... .......... .......... .......... 25% 180M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19100K .......... .......... .......... .......... .......... 26% 174M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19150K .......... .......... .......... .......... .......... 26% 166M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19200K .......... .......... .......... .......... .......... 26% 212M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19250K .......... .......... .......... .......... .......... 26% 176M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19300K .......... .......... .......... .......... .......... 26% 185M 0s 2019-12-09T10:08:53.309 INFO:teuthology.orchestra.run.smithi059.stderr: 19350K .......... .......... .......... .......... .......... 26% 188M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19400K .......... .......... .......... .......... .......... 26% 199M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19450K .......... .......... .......... .......... .......... 26% 223M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19500K .......... .......... .......... .......... .......... 26% 177M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19550K .......... .......... .......... .......... .......... 26% 194M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19600K .......... .......... .......... .......... .......... 26% 257M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19650K .......... .......... .......... .......... .......... 26% 368M 0s 2019-12-09T10:08:53.310 INFO:teuthology.orchestra.run.smithi059.stderr: 19700K .......... .......... .......... .......... .......... 26% 352M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 19750K .......... .......... .......... .......... .......... 26% 302M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 19800K .......... .......... .......... .......... .......... 26% 293M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 19850K .......... .......... .......... .......... .......... 27% 207M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 19900K .......... .......... .......... .......... .......... 27% 229M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 19950K .......... .......... .......... .......... .......... 27% 201M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 20000K .......... .......... .......... .......... .......... 27% 238M 0s 2019-12-09T10:08:53.311 INFO:teuthology.orchestra.run.smithi059.stderr: 20050K .......... .......... .......... .......... .......... 27% 244M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20100K .......... .......... .......... .......... .......... 27% 247M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20150K .......... .......... .......... .......... .......... 27% 187M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20200K .......... .......... .......... .......... .......... 27% 261M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20250K .......... .......... .......... .......... .......... 27% 190M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20300K .......... .......... .......... .......... .......... 27% 186M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20350K .......... .......... .......... .......... .......... 27% 200M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20400K .......... .......... .......... .......... .......... 27% 214M 0s 2019-12-09T10:08:53.312 INFO:teuthology.orchestra.run.smithi059.stderr: 20450K .......... .......... .......... .......... .......... 27% 199M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20500K .......... .......... .......... .......... .......... 27% 173M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20550K .......... .......... .......... .......... .......... 28% 158M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20600K .......... .......... .......... .......... .......... 28% 190M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20650K .......... .......... .......... .......... .......... 28% 186M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20700K .......... .......... .......... .......... .......... 28% 183M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20750K .......... .......... .......... .......... .......... 28% 149M 0s 2019-12-09T10:08:53.313 INFO:teuthology.orchestra.run.smithi059.stderr: 20800K .......... .......... .......... .......... .......... 28% 211M 0s 2019-12-09T10:08:53.315 INFO:teuthology.orchestra.run.smithi059.stderr: 20850K .......... .......... .......... .......... .......... 28% 194M 0s 2019-12-09T10:08:53.315 INFO:teuthology.orchestra.run.smithi059.stderr: 20900K .......... .......... .......... .......... .......... 28% 203M 0s 2019-12-09T10:08:53.315 INFO:teuthology.orchestra.run.smithi059.stderr: 20950K .......... .......... .......... .......... .......... 28% 158M 0s 2019-12-09T10:08:53.315 INFO:teuthology.orchestra.run.smithi059.stderr: 21000K .......... .......... .......... .......... .......... 28% 213M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21050K .......... .......... .......... .......... .......... 28% 197M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21100K .......... .......... .......... .......... .......... 28% 216M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21150K .......... .......... .......... .......... .......... 28% 227M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21200K .......... .......... .......... .......... .......... 28% 285M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21250K .......... .......... .......... .......... .......... 28% 347M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21300K .......... .......... .......... .......... .......... 29% 272M 0s 2019-12-09T10:08:53.316 INFO:teuthology.orchestra.run.smithi059.stderr: 21350K .......... .......... .......... .......... .......... 29% 150M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21400K .......... .......... .......... .......... .......... 29% 179M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21450K .......... .......... .......... .......... .......... 29% 204M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21500K .......... .......... .......... .......... .......... 29% 181M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21550K .......... .......... .......... .......... .......... 29% 215M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21600K .......... .......... .......... .......... .......... 29% 241M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21650K .......... .......... .......... .......... .......... 29% 252M 0s 2019-12-09T10:08:53.317 INFO:teuthology.orchestra.run.smithi059.stderr: 21700K .......... .......... .......... .......... .......... 29% 255M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 21750K .......... .......... .......... .......... .......... 29% 140M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 21800K .......... .......... .......... .......... .......... 29% 176M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 21850K .......... .......... .......... .......... .......... 29% 163M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 21900K .......... .......... .......... .......... .......... 29% 173M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 21950K .......... .......... .......... .......... .......... 29% 270M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 22000K .......... .......... .......... .......... .......... 29% 330M 0s 2019-12-09T10:08:53.318 INFO:teuthology.orchestra.run.smithi059.stderr: 22050K .......... .......... .......... .......... .......... 30% 275M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22100K .......... .......... .......... .......... .......... 30% 241M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22150K .......... .......... .......... .......... .......... 30% 184M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22200K .......... .......... .......... .......... .......... 30% 208M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22250K .......... .......... .......... .......... .......... 30% 315M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22300K .......... .......... .......... .......... .......... 30% 227M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22350K .......... .......... .......... .......... .......... 30% 219M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22400K .......... .......... .......... .......... .......... 30% 263M 0s 2019-12-09T10:08:53.319 INFO:teuthology.orchestra.run.smithi059.stderr: 22450K .......... .......... .......... .......... .......... 30% 241M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22500K .......... .......... .......... .......... .......... 30% 351M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22550K .......... .......... .......... .......... .......... 30% 184M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22600K .......... .......... .......... .......... .......... 30% 175M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22650K .......... .......... .......... .......... .......... 30% 282M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22700K .......... .......... .......... .......... .......... 30% 332M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22750K .......... .......... .......... .......... .......... 30% 287M 0s 2019-12-09T10:08:53.320 INFO:teuthology.orchestra.run.smithi059.stderr: 22800K .......... .......... .......... .......... .......... 31% 348M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 22850K .......... .......... .......... .......... .......... 31% 279M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 22900K .......... .......... .......... .......... .......... 31% 285M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 22950K .......... .......... .......... .......... .......... 31% 210M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 23000K .......... .......... .......... .......... .......... 31% 353M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 23050K .......... .......... .......... .......... .......... 31% 286M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 23100K .......... .......... .......... .......... .......... 31% 338M 0s 2019-12-09T10:08:53.321 INFO:teuthology.orchestra.run.smithi059.stderr: 23150K .......... .......... .......... .......... .......... 31% 305M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23200K .......... .......... .......... .......... .......... 31% 287M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23250K .......... .......... .......... .......... .......... 31% 177M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23300K .......... .......... .......... .......... .......... 31% 218M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23350K .......... .......... .......... .......... .......... 31% 183M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23400K .......... .......... .......... .......... .......... 31% 245M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23450K .......... .......... .......... .......... .......... 31% 254M 0s 2019-12-09T10:08:53.322 INFO:teuthology.orchestra.run.smithi059.stderr: 23500K .......... .......... .......... .......... .......... 32% 221M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23550K .......... .......... .......... .......... .......... 32% 191M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23600K .......... .......... .......... .......... .......... 32% 194M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23650K .......... .......... .......... .......... .......... 32% 189M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23700K .......... .......... .......... .......... .......... 32% 201M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23750K .......... .......... .......... .......... .......... 32% 160M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23800K .......... .......... .......... .......... .......... 32% 201M 0s 2019-12-09T10:08:53.323 INFO:teuthology.orchestra.run.smithi059.stderr: 23850K .......... .......... .......... .......... .......... 32% 196M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 23900K .......... .......... .......... .......... .......... 32% 159M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 23950K .......... .......... .......... .......... .......... 32% 164M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 24000K .......... .......... .......... .......... .......... 32% 202M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 24050K .......... .......... .......... .......... .......... 32% 204M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 24100K .......... .......... .......... .......... .......... 32% 241M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 24150K .......... .......... .......... .......... .......... 32% 192M 0s 2019-12-09T10:08:53.324 INFO:teuthology.orchestra.run.smithi059.stderr: 24200K .......... .......... .......... .......... .......... 32% 216M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24250K .......... .......... .......... .......... .......... 33% 202M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24300K .......... .......... .......... .......... .......... 33% 258M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24350K .......... .......... .......... .......... .......... 33% 163M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24400K .......... .......... .......... .......... .......... 33% 216M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24450K .......... .......... .......... .......... .......... 33% 229M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24500K .......... .......... .......... .......... .......... 33% 254M 0s 2019-12-09T10:08:53.325 INFO:teuthology.orchestra.run.smithi059.stderr: 24550K .......... .......... .......... .......... .......... 33% 193M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24600K .......... .......... .......... .......... .......... 33% 203M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24650K .......... .......... .......... .......... .......... 33% 216M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24700K .......... .......... .......... .......... .......... 33% 216M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24750K .......... .......... .......... .......... .......... 33% 181M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24800K .......... .......... .......... .......... .......... 33% 221M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24850K .......... .......... .......... .......... .......... 33% 210M 0s 2019-12-09T10:08:53.326 INFO:teuthology.orchestra.run.smithi059.stderr: 24900K .......... .......... .......... .......... .......... 33% 200M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 24950K .......... .......... .......... .......... .......... 33% 230M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25000K .......... .......... .......... .......... .......... 34% 255M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25050K .......... .......... .......... .......... .......... 34% 177M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25100K .......... .......... .......... .......... .......... 34% 209M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25150K .......... .......... .......... .......... .......... 34% 206M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25200K .......... .......... .......... .......... .......... 34% 189M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25250K .......... .......... .......... .......... .......... 34% 238M 0s 2019-12-09T10:08:53.327 INFO:teuthology.orchestra.run.smithi059.stderr: 25300K .......... .......... .......... .......... .......... 34% 224M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25350K .......... .......... .......... .......... .......... 34% 231M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25400K .......... .......... .......... .......... .......... 34% 236M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25450K .......... .......... .......... .......... .......... 34% 206M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25500K .......... .......... .......... .......... .......... 34% 206M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25550K .......... .......... .......... .......... .......... 34% 168M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25600K .......... .......... .......... .......... .......... 34% 194M 0s 2019-12-09T10:08:53.328 INFO:teuthology.orchestra.run.smithi059.stderr: 25650K .......... .......... .......... .......... .......... 34% 217M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 25700K .......... .......... .......... .......... .......... 35% 216M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 25750K .......... .......... .......... .......... .......... 35% 178M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 25800K .......... .......... .......... .......... .......... 35% 195M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 25850K .......... .......... .......... .......... .......... 35% 222M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 25900K .......... .......... .......... .......... .......... 35% 203M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 25950K .......... .......... .......... .......... .......... 35% 190M 0s 2019-12-09T10:08:53.329 INFO:teuthology.orchestra.run.smithi059.stderr: 26000K .......... .......... .......... .......... .......... 35% 205M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26050K .......... .......... .......... .......... .......... 35% 274M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26100K .......... .......... .......... .......... .......... 35% 198M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26150K .......... .......... .......... .......... .......... 35% 189M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26200K .......... .......... .......... .......... .......... 35% 209M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26250K .......... .......... .......... .......... .......... 35% 206M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26300K .......... .......... .......... .......... .......... 35% 217M 0s 2019-12-09T10:08:53.330 INFO:teuthology.orchestra.run.smithi059.stderr: 26350K .......... .......... .......... .......... .......... 35% 193M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26400K .......... .......... .......... .......... .......... 35% 212M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26450K .......... .......... .......... .......... .......... 36% 189M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26500K .......... .......... .......... .......... .......... 36% 234M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26550K .......... .......... .......... .......... .......... 36% 246M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26600K .......... .......... .......... .......... .......... 36% 203M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26650K .......... .......... .......... .......... .......... 36% 202M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26700K .......... .......... .......... .......... .......... 36% 234M 0s 2019-12-09T10:08:53.331 INFO:teuthology.orchestra.run.smithi059.stderr: 26750K .......... .......... .......... .......... .......... 36% 217M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 26800K .......... .......... .......... .......... .......... 36% 223M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 26850K .......... .......... .......... .......... .......... 36% 217M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 26900K .......... .......... .......... .......... .......... 36% 210M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 26950K .......... .......... .......... .......... .......... 36% 237M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 27000K .......... .......... .......... .......... .......... 36% 224M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 27050K .......... .......... .......... .......... .......... 36% 202M 0s 2019-12-09T10:08:53.332 INFO:teuthology.orchestra.run.smithi059.stderr: 27100K .......... .......... .......... .......... .......... 36% 229M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27150K .......... .......... .......... .......... .......... 36% 217M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27200K .......... .......... .......... .......... .......... 37% 214M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27250K .......... .......... .......... .......... .......... 37% 207M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27300K .......... .......... .......... .......... .......... 37% 284M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27350K .......... .......... .......... .......... .......... 37% 173M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27400K .......... .......... .......... .......... .......... 37% 231M 0s 2019-12-09T10:08:53.333 INFO:teuthology.orchestra.run.smithi059.stderr: 27450K .......... .......... .......... .......... .......... 37% 226M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27500K .......... .......... .......... .......... .......... 37% 222M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27550K .......... .......... .......... .......... .......... 37% 183M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27600K .......... .......... .......... .......... .......... 37% 249M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27650K .......... .......... .......... .......... .......... 37% 194M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27700K .......... .......... .......... .......... .......... 37% 233M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27750K .......... .......... .......... .......... .......... 37% 205M 0s 2019-12-09T10:08:53.334 INFO:teuthology.orchestra.run.smithi059.stderr: 27800K .......... .......... .......... .......... .......... 37% 233M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 27850K .......... .......... .......... .......... .......... 37% 235M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 27900K .......... .......... .......... .......... .......... 37% 231M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 27950K .......... .......... .......... .......... .......... 38% 208M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 28000K .......... .......... .......... .......... .......... 38% 223M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 28050K .......... .......... .......... .......... .......... 38% 236M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 28100K .......... .......... .......... .......... .......... 38% 205M 0s 2019-12-09T10:08:53.335 INFO:teuthology.orchestra.run.smithi059.stderr: 28150K .......... .......... .......... .......... .......... 38% 187M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28200K .......... .......... .......... .......... .......... 38% 275M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28250K .......... .......... .......... .......... .......... 38% 226M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28300K .......... .......... .......... .......... .......... 38% 230M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28350K .......... .......... .......... .......... .......... 38% 174M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28400K .......... .......... .......... .......... .......... 38% 210M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28450K .......... .......... .......... .......... .......... 38% 217M 0s 2019-12-09T10:08:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: 28500K .......... .......... .......... .......... .......... 38% 269M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28550K .......... .......... .......... .......... .......... 38% 197M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28600K .......... .......... .......... .......... .......... 38% 273M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28650K .......... .......... .......... .......... .......... 39% 214M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28700K .......... .......... .......... .......... .......... 39% 231M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28750K .......... .......... .......... .......... .......... 39% 198M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28800K .......... .......... .......... .......... .......... 39% 195M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28850K .......... .......... .......... .......... .......... 39% 278M 0s 2019-12-09T10:08:53.337 INFO:teuthology.orchestra.run.smithi059.stderr: 28900K .......... .......... .......... .......... .......... 39% 232M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 28950K .......... .......... .......... .......... .......... 39% 245M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 29000K .......... .......... .......... .......... .......... 39% 243M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 29050K .......... .......... .......... .......... .......... 39% 202M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 29100K .......... .......... .......... .......... .......... 39% 224M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 29150K .......... .......... .......... .......... .......... 39% 228M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 29200K .......... .......... .......... .......... .......... 39% 246M 0s 2019-12-09T10:08:53.338 INFO:teuthology.orchestra.run.smithi059.stderr: 29250K .......... .......... .......... .......... .......... 39% 222M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29300K .......... .......... .......... .......... .......... 39% 252M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29350K .......... .......... .......... .......... .......... 39% 202M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29400K .......... .......... .......... .......... .......... 40% 267M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29450K .......... .......... .......... .......... .......... 40% 216M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29500K .......... .......... .......... .......... .......... 40% 245M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29550K .......... .......... .......... .......... .......... 40% 212M 0s 2019-12-09T10:08:53.339 INFO:teuthology.orchestra.run.smithi059.stderr: 29600K .......... .......... .......... .......... .......... 40% 238M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29650K .......... .......... .......... .......... .......... 40% 255M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29700K .......... .......... .......... .......... .......... 40% 257M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29750K .......... .......... .......... .......... .......... 40% 233M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29800K .......... .......... .......... .......... .......... 40% 212M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29850K .......... .......... .......... .......... .......... 40% 223M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29900K .......... .......... .......... .......... .......... 40% 264M 0s 2019-12-09T10:08:53.340 INFO:teuthology.orchestra.run.smithi059.stderr: 29950K .......... .......... .......... .......... .......... 40% 182M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30000K .......... .......... .......... .......... .......... 40% 223M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30050K .......... .......... .......... .......... .......... 40% 227M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30100K .......... .......... .......... .......... .......... 40% 245M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30150K .......... .......... .......... .......... .......... 41% 200M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30200K .......... .......... .......... .......... .......... 41% 239M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30250K .......... .......... .......... .......... .......... 41% 212M 0s 2019-12-09T10:08:53.341 INFO:teuthology.orchestra.run.smithi059.stderr: 30300K .......... .......... .......... .......... .......... 41% 270M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30350K .......... .......... .......... .......... .......... 41% 189M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30400K .......... .......... .......... .......... .......... 41% 247M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30450K .......... .......... .......... .......... .......... 41% 219M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30500K .......... .......... .......... .......... .......... 41% 241M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30550K .......... .......... .......... .......... .......... 41% 229M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30600K .......... .......... .......... .......... .......... 41% 290M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30650K .......... .......... .......... .......... .......... 41% 222M 0s 2019-12-09T10:08:53.342 INFO:teuthology.orchestra.run.smithi059.stderr: 30700K .......... .......... .......... .......... .......... 41% 245M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 30750K .......... .......... .......... .......... .......... 41% 184M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 30800K .......... .......... .......... .......... .......... 41% 252M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 30850K .......... .......... .......... .......... .......... 42% 221M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 30900K .......... .......... .......... .......... .......... 42% 238M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 30950K .......... .......... .......... .......... .......... 42% 199M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 31000K .......... .......... .......... .......... .......... 42% 229M 0s 2019-12-09T10:08:53.343 INFO:teuthology.orchestra.run.smithi059.stderr: 31050K .......... .......... .......... .......... .......... 42% 243M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31100K .......... .......... .......... .......... .......... 42% 237M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31150K .......... .......... .......... .......... .......... 42% 211M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31200K .......... .......... .......... .......... .......... 42% 242M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31250K .......... .......... .......... .......... .......... 42% 284M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31300K .......... .......... .......... .......... .......... 42% 230M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31350K .......... .......... .......... .......... .......... 42% 195M 0s 2019-12-09T10:08:53.344 INFO:teuthology.orchestra.run.smithi059.stderr: 31400K .......... .......... .......... .......... .......... 42% 271M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31450K .......... .......... .......... .......... .......... 42% 242M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31500K .......... .......... .......... .......... .......... 42% 201M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31550K .......... .......... .......... .......... .......... 42% 209M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31600K .......... .......... .......... .......... .......... 43% 219M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31650K .......... .......... .......... .......... .......... 43% 234M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31700K .......... .......... .......... .......... .......... 43% 220M 0s 2019-12-09T10:08:53.345 INFO:teuthology.orchestra.run.smithi059.stderr: 31750K .......... .......... .......... .......... .......... 43% 187M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 31800K .......... .......... .......... .......... .......... 43% 253M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 31850K .......... .......... .......... .......... .......... 43% 221M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 31900K .......... .......... .......... .......... .......... 43% 222M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 31950K .......... .......... .......... .......... .......... 43% 208M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 32000K .......... .......... .......... .......... .......... 43% 277M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 32050K .......... .......... .......... .......... .......... 43% 246M 0s 2019-12-09T10:08:53.346 INFO:teuthology.orchestra.run.smithi059.stderr: 32100K .......... .......... .......... .......... .......... 43% 183M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32150K .......... .......... .......... .......... .......... 43% 165M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32200K .......... .......... .......... .......... .......... 43% 182M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32250K .......... .......... .......... .......... .......... 43% 273M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32300K .......... .......... .......... .......... .......... 43% 217M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32350K .......... .......... .......... .......... .......... 44% 196M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32400K .......... .......... .......... .......... .......... 44% 230M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32450K .......... .......... .......... .......... .......... 44% 204M 0s 2019-12-09T10:08:53.347 INFO:teuthology.orchestra.run.smithi059.stderr: 32500K .......... .......... .......... .......... .......... 44% 227M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32550K .......... .......... .......... .......... .......... 44% 183M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32600K .......... .......... .......... .......... .......... 44% 255M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32650K .......... .......... .......... .......... .......... 44% 234M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32700K .......... .......... .......... .......... .......... 44% 236M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32750K .......... .......... .......... .......... .......... 44% 190M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32800K .......... .......... .......... .......... .......... 44% 222M 0s 2019-12-09T10:08:53.348 INFO:teuthology.orchestra.run.smithi059.stderr: 32850K .......... .......... .......... .......... .......... 44% 256M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 32900K .......... .......... .......... .......... .......... 44% 293M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 32950K .......... .......... .......... .......... .......... 44% 169M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 33000K .......... .......... .......... .......... .......... 44% 197M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 33050K .......... .......... .......... .......... .......... 44% 274M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 33100K .......... .......... .......... .......... .......... 45% 260M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 33150K .......... .......... .......... .......... .......... 45% 198M 0s 2019-12-09T10:08:53.349 INFO:teuthology.orchestra.run.smithi059.stderr: 33200K .......... .......... .......... .......... .......... 45% 186M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33250K .......... .......... .......... .......... .......... 45% 226M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33300K .......... .......... .......... .......... .......... 45% 269M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33350K .......... .......... .......... .......... .......... 45% 169M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33400K .......... .......... .......... .......... .......... 45% 258M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33450K .......... .......... .......... .......... .......... 45% 240M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33500K .......... .......... .......... .......... .......... 45% 228M 0s 2019-12-09T10:08:53.350 INFO:teuthology.orchestra.run.smithi059.stderr: 33550K .......... .......... .......... .......... .......... 45% 182M 0s 2019-12-09T10:08:53.351 INFO:teuthology.orchestra.run.smithi059.stderr: 33600K .......... .......... .......... .......... .......... 45% 236M 0s 2019-12-09T10:08:53.351 INFO:teuthology.orchestra.run.smithi059.stderr: 33650K .......... .......... .......... .......... .......... 45% 273M 0s 2019-12-09T10:08:53.351 INFO:teuthology.orchestra.run.smithi059.stderr: 33700K .......... .......... .......... .......... .......... 45% 240M 0s 2019-12-09T10:08:53.351 INFO:teuthology.orchestra.run.smithi059.stderr: 33750K .......... .......... .......... .......... .......... 45% 173M 0s 2019-12-09T10:08:53.351 INFO:teuthology.orchestra.run.smithi059.stderr: 33800K .......... .......... .......... .......... .......... 46% 205M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 33850K .......... .......... .......... .......... .......... 46% 201M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 33900K .......... .......... .......... .......... .......... 46% 245M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 33950K .......... .......... .......... .......... .......... 46% 198M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 34000K .......... .......... .......... .......... .......... 46% 200M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 34050K .......... .......... .......... .......... .......... 46% 203M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 34100K .......... .......... .......... .......... .......... 46% 262M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 34150K .......... .......... .......... .......... .......... 46% 198M 0s 2019-12-09T10:08:53.352 INFO:teuthology.orchestra.run.smithi059.stderr: 34200K .......... .......... .......... .......... .......... 46% 231M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34250K .......... .......... .......... .......... .......... 46% 246M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34300K .......... .......... .......... .......... .......... 46% 210M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34350K .......... .......... .......... .......... .......... 46% 161M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34400K .......... .......... .......... .......... .......... 46% 266M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34450K .......... .......... .......... .......... .......... 46% 229M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34500K .......... .......... .......... .......... .......... 46% 236M 0s 2019-12-09T10:08:53.353 INFO:teuthology.orchestra.run.smithi059.stderr: 34550K .......... .......... .......... .......... .......... 47% 249M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34600K .......... .......... .......... .......... .......... 47% 231M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34650K .......... .......... .......... .......... .......... 47% 221M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34700K .......... .......... .......... .......... .......... 47% 236M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34750K .......... .......... .......... .......... .......... 47% 186M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34800K .......... .......... .......... .......... .......... 47% 186M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34850K .......... .......... .......... .......... .......... 47% 220M 0s 2019-12-09T10:08:53.354 INFO:teuthology.orchestra.run.smithi059.stderr: 34900K .......... .......... .......... .......... .......... 47% 206M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 34950K .......... .......... .......... .......... .......... 47% 239M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 35000K .......... .......... .......... .......... .......... 47% 212M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 35050K .......... .......... .......... .......... .......... 47% 213M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 35100K .......... .......... .......... .......... .......... 47% 225M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 35150K .......... .......... .......... .......... .......... 47% 210M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 35200K .......... .......... .......... .......... .......... 47% 226M 0s 2019-12-09T10:08:53.355 INFO:teuthology.orchestra.run.smithi059.stderr: 35250K .......... .......... .......... .......... .......... 47% 233M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35300K .......... .......... .......... .......... .......... 48% 248M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35350K .......... .......... .......... .......... .......... 48% 205M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35400K .......... .......... .......... .......... .......... 48% 188M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35450K .......... .......... .......... .......... .......... 48% 235M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35500K .......... .......... .......... .......... .......... 48% 239M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35550K .......... .......... .......... .......... .......... 48% 222M 0s 2019-12-09T10:08:53.356 INFO:teuthology.orchestra.run.smithi059.stderr: 35600K .......... .......... .......... .......... .......... 48% 250M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35650K .......... .......... .......... .......... .......... 48% 230M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35700K .......... .......... .......... .......... .......... 48% 254M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35750K .......... .......... .......... .......... .......... 48% 220M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35800K .......... .......... .......... .......... .......... 48% 240M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35850K .......... .......... .......... .......... .......... 48% 202M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35900K .......... .......... .......... .......... .......... 48% 219M 0s 2019-12-09T10:08:53.357 INFO:teuthology.orchestra.run.smithi059.stderr: 35950K .......... .......... .......... .......... .......... 48% 196M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36000K .......... .......... .......... .......... .......... 49% 191M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36050K .......... .......... .......... .......... .......... 49% 234M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36100K .......... .......... .......... .......... .......... 49% 221M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36150K .......... .......... .......... .......... .......... 49% 188M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36200K .......... .......... .......... .......... .......... 49% 285M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36250K .......... .......... .......... .......... .......... 49% 258M 0s 2019-12-09T10:08:53.358 INFO:teuthology.orchestra.run.smithi059.stderr: 36300K .......... .......... .......... .......... .......... 49% 211M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36350K .......... .......... .......... .......... .......... 49% 182M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36400K .......... .......... .......... .......... .......... 49% 205M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36450K .......... .......... .......... .......... .......... 49% 221M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36500K .......... .......... .......... .......... .......... 49% 238M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36550K .......... .......... .......... .......... .......... 49% 193M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36600K .......... .......... .......... .......... .......... 49% 245M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36650K .......... .......... .......... .......... .......... 49% 220M 0s 2019-12-09T10:08:53.359 INFO:teuthology.orchestra.run.smithi059.stderr: 36700K .......... .......... .......... .......... .......... 49% 211M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 36750K .......... .......... .......... .......... .......... 50% 165M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 36800K .......... .......... .......... .......... .......... 50% 264M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 36850K .......... .......... .......... .......... .......... 50% 205M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 36900K .......... .......... .......... .......... .......... 50% 233M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 36950K .......... .......... .......... .......... .......... 50% 182M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 37000K .......... .......... .......... .......... .......... 50% 231M 0s 2019-12-09T10:08:53.360 INFO:teuthology.orchestra.run.smithi059.stderr: 37050K .......... .......... .......... .......... .......... 50% 203M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37100K .......... .......... .......... .......... .......... 50% 245M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37150K .......... .......... .......... .......... .......... 50% 175M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37200K .......... .......... .......... .......... .......... 50% 241M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37250K .......... .......... .......... .......... .......... 50% 201M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37300K .......... .......... .......... .......... .......... 50% 207M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37350K .......... .......... .......... .......... .......... 50% 191M 0s 2019-12-09T10:08:53.361 INFO:teuthology.orchestra.run.smithi059.stderr: 37400K .......... .......... .......... .......... .......... 50% 260M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37450K .......... .......... .......... .......... .......... 50% 226M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37500K .......... .......... .......... .......... .......... 51% 221M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37550K .......... .......... .......... .......... .......... 51% 206M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37600K .......... .......... .......... .......... .......... 51% 187M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37650K .......... .......... .......... .......... .......... 51% 210M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37700K .......... .......... .......... .......... .......... 51% 265M 0s 2019-12-09T10:08:53.362 INFO:teuthology.orchestra.run.smithi059.stderr: 37750K .......... .......... .......... .......... .......... 51% 176M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 37800K .......... .......... .......... .......... .......... 51% 219M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 37850K .......... .......... .......... .......... .......... 51% 255M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 37900K .......... .......... .......... .......... .......... 51% 213M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 37950K .......... .......... .......... .......... .......... 51% 194M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 38000K .......... .......... .......... .......... .......... 51% 226M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 38050K .......... .......... .......... .......... .......... 51% 220M 0s 2019-12-09T10:08:53.363 INFO:teuthology.orchestra.run.smithi059.stderr: 38100K .......... .......... .......... .......... .......... 51% 201M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38150K .......... .......... .......... .......... .......... 51% 229M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38200K .......... .......... .......... .......... .......... 51% 226M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38250K .......... .......... .......... .......... .......... 52% 187M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38300K .......... .......... .......... .......... .......... 52% 223M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38350K .......... .......... .......... .......... .......... 52% 197M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38400K .......... .......... .......... .......... .......... 52% 292M 0s 2019-12-09T10:08:53.364 INFO:teuthology.orchestra.run.smithi059.stderr: 38450K .......... .......... .......... .......... .......... 52% 231M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38500K .......... .......... .......... .......... .......... 52% 219M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38550K .......... .......... .......... .......... .......... 52% 185M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38600K .......... .......... .......... .......... .......... 52% 277M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38650K .......... .......... .......... .......... .......... 52% 244M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38700K .......... .......... .......... .......... .......... 52% 206M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38750K .......... .......... .......... .......... .......... 52% 211M 0s 2019-12-09T10:08:53.365 INFO:teuthology.orchestra.run.smithi059.stderr: 38800K .......... .......... .......... .......... .......... 52% 204M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 38850K .......... .......... .......... .......... .......... 52% 238M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 38900K .......... .......... .......... .......... .......... 52% 238M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 38950K .......... .......... .......... .......... .......... 53% 203M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 39000K .......... .......... .......... .......... .......... 53% 274M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 39050K .......... .......... .......... .......... .......... 53% 239M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 39100K .......... .......... .......... .......... .......... 53% 229M 0s 2019-12-09T10:08:53.366 INFO:teuthology.orchestra.run.smithi059.stderr: 39150K .......... .......... .......... .......... .......... 53% 194M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39200K .......... .......... .......... .......... .......... 53% 274M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39250K .......... .......... .......... .......... .......... 53% 227M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39300K .......... .......... .......... .......... .......... 53% 222M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39350K .......... .......... .......... .......... .......... 53% 200M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39400K .......... .......... .......... .......... .......... 53% 234M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39450K .......... .......... .......... .......... .......... 53% 214M 0s 2019-12-09T10:08:53.367 INFO:teuthology.orchestra.run.smithi059.stderr: 39500K .......... .......... .......... .......... .......... 53% 268M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39550K .......... .......... .......... .......... .......... 53% 182M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39600K .......... .......... .......... .......... .......... 53% 214M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39650K .......... .......... .......... .......... .......... 53% 279M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39700K .......... .......... .......... .......... .......... 54% 236M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39750K .......... .......... .......... .......... .......... 54% 182M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39800K .......... .......... .......... .......... .......... 54% 223M 0s 2019-12-09T10:08:53.368 INFO:teuthology.orchestra.run.smithi059.stderr: 39850K .......... .......... .......... .......... .......... 54% 241M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 39900K .......... .......... .......... .......... .......... 54% 230M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 39950K .......... .......... .......... .......... .......... 54% 202M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 40000K .......... .......... .......... .......... .......... 54% 224M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 40050K .......... .......... .......... .......... .......... 54% 215M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 40100K .......... .......... .......... .......... .......... 54% 263M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 40150K .......... .......... .......... .......... .......... 54% 185M 0s 2019-12-09T10:08:53.369 INFO:teuthology.orchestra.run.smithi059.stderr: 40200K .......... .......... .......... .......... .......... 54% 229M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40250K .......... .......... .......... .......... .......... 54% 286M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40300K .......... .......... .......... .......... .......... 54% 192M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40350K .......... .......... .......... .......... .......... 54% 212M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40400K .......... .......... .......... .......... .......... 54% 249M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40450K .......... .......... .......... .......... .......... 55% 265M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40500K .......... .......... .......... .......... .......... 55% 219M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40550K .......... .......... .......... .......... .......... 55% 193M 0s 2019-12-09T10:08:53.370 INFO:teuthology.orchestra.run.smithi059.stderr: 40600K .......... .......... .......... .......... .......... 55% 245M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40650K .......... .......... .......... .......... .......... 55% 258M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40700K .......... .......... .......... .......... .......... 55% 268M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40750K .......... .......... .......... .......... .......... 55% 184M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40800K .......... .......... .......... .......... .......... 55% 257M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40850K .......... .......... .......... .......... .......... 55% 234M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40900K .......... .......... .......... .......... .......... 55% 267M 0s 2019-12-09T10:08:53.371 INFO:teuthology.orchestra.run.smithi059.stderr: 40950K .......... .......... .......... .......... .......... 55% 232M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41000K .......... .......... .......... .......... .......... 55% 213M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41050K .......... .......... .......... .......... .......... 55% 230M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41100K .......... .......... .......... .......... .......... 55% 257M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41150K .......... .......... .......... .......... .......... 56% 225M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41200K .......... .......... .......... .......... .......... 56% 246M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41250K .......... .......... .......... .......... .......... 56% 257M 0s 2019-12-09T10:08:53.372 INFO:teuthology.orchestra.run.smithi059.stderr: 41300K .......... .......... .......... .......... .......... 56% 203M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41350K .......... .......... .......... .......... .......... 56% 213M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41400K .......... .......... .......... .......... .......... 56% 264M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41450K .......... .......... .......... .......... .......... 56% 214M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41500K .......... .......... .......... .......... .......... 56% 238M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41550K .......... .......... .......... .......... .......... 56% 206M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41600K .......... .......... .......... .......... .......... 56% 235M 0s 2019-12-09T10:08:53.373 INFO:teuthology.orchestra.run.smithi059.stderr: 41650K .......... .......... .......... .......... .......... 56% 272M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 41700K .......... .......... .......... .......... .......... 56% 238M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 41750K .......... .......... .......... .......... .......... 56% 166M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 41800K .......... .......... .......... .......... .......... 56% 225M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 41850K .......... .......... .......... .......... .......... 56% 247M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 41900K .......... .......... .......... .......... .......... 57% 225M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 41950K .......... .......... .......... .......... .......... 57% 190M 0s 2019-12-09T10:08:53.374 INFO:teuthology.orchestra.run.smithi059.stderr: 42000K .......... .......... .......... .......... .......... 57% 236M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42050K .......... .......... .......... .......... .......... 57% 235M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42100K .......... .......... .......... .......... .......... 57% 212M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42150K .......... .......... .......... .......... .......... 57% 220M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42200K .......... .......... .......... .......... .......... 57% 238M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42250K .......... .......... .......... .......... .......... 57% 232M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42300K .......... .......... .......... .......... .......... 57% 247M 0s 2019-12-09T10:08:53.375 INFO:teuthology.orchestra.run.smithi059.stderr: 42350K .......... .......... .......... .......... .......... 57% 199M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42400K .......... .......... .......... .......... .......... 57% 241M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42450K .......... .......... .......... .......... .......... 57% 236M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42500K .......... .......... .......... .......... .......... 57% 276M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42550K .......... .......... .......... .......... .......... 57% 203M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42600K .......... .......... .......... .......... .......... 57% 216M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42650K .......... .......... .......... .......... .......... 58% 263M 0s 2019-12-09T10:08:53.376 INFO:teuthology.orchestra.run.smithi059.stderr: 42700K .......... .......... .......... .......... .......... 58% 272M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 42750K .......... .......... .......... .......... .......... 58% 175M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 42800K .......... .......... .......... .......... .......... 58% 291M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 42850K .......... .......... .......... .......... .......... 58% 235M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 42900K .......... .......... .......... .......... .......... 58% 217M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 42950K .......... .......... .......... .......... .......... 58% 186M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 43000K .......... .......... .......... .......... .......... 58% 232M 0s 2019-12-09T10:08:53.377 INFO:teuthology.orchestra.run.smithi059.stderr: 43050K .......... .......... .......... .......... .......... 58% 223M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43100K .......... .......... .......... .......... .......... 58% 232M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43150K .......... .......... .......... .......... .......... 58% 202M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43200K .......... .......... .......... .......... .......... 58% 244M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43250K .......... .......... .......... .......... .......... 58% 263M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43300K .......... .......... .......... .......... .......... 58% 192M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43350K .......... .......... .......... .......... .......... 59% 226M 0s 2019-12-09T10:08:53.378 INFO:teuthology.orchestra.run.smithi059.stderr: 43400K .......... .......... .......... .......... .......... 59% 230M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43450K .......... .......... .......... .......... .......... 59% 240M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43500K .......... .......... .......... .......... .......... 59% 253M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43550K .......... .......... .......... .......... .......... 59% 212M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43600K .......... .......... .......... .......... .......... 59% 225M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43650K .......... .......... .......... .......... .......... 59% 223M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43700K .......... .......... .......... .......... .......... 59% 229M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43750K .......... .......... .......... .......... .......... 59% 210M 0s 2019-12-09T10:08:53.379 INFO:teuthology.orchestra.run.smithi059.stderr: 43800K .......... .......... .......... .......... .......... 59% 261M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 43850K .......... .......... .......... .......... .......... 59% 204M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 43900K .......... .......... .......... .......... .......... 59% 223M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 43950K .......... .......... .......... .......... .......... 59% 195M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 44000K .......... .......... .......... .......... .......... 59% 215M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 44050K .......... .......... .......... .......... .......... 59% 217M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 44100K .......... .......... .......... .......... .......... 60% 244M 0s 2019-12-09T10:08:53.380 INFO:teuthology.orchestra.run.smithi059.stderr: 44150K .......... .......... .......... .......... .......... 60% 206M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44200K .......... .......... .......... .......... .......... 60% 276M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44250K .......... .......... .......... .......... .......... 60% 175M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44300K .......... .......... .......... .......... .......... 60% 202M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44350K .......... .......... .......... .......... .......... 60% 183M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44400K .......... .......... .......... .......... .......... 60% 216M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44450K .......... .......... .......... .......... .......... 60% 289M 0s 2019-12-09T10:08:53.381 INFO:teuthology.orchestra.run.smithi059.stderr: 44500K .......... .......... .......... .......... .......... 60% 205M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44550K .......... .......... .......... .......... .......... 60% 173M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44600K .......... .......... .......... .......... .......... 60% 234M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44650K .......... .......... .......... .......... .......... 60% 233M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44700K .......... .......... .......... .......... .......... 60% 210M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44750K .......... .......... .......... .......... .......... 60% 168M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44800K .......... .......... .......... .......... .......... 60% 232M 0s 2019-12-09T10:08:53.382 INFO:teuthology.orchestra.run.smithi059.stderr: 44850K .......... .......... .......... .......... .......... 61% 229M 0s 2019-12-09T10:08:53.383 INFO:teuthology.orchestra.run.smithi059.stderr: 44900K .......... .......... .......... .......... .......... 61% 250M 0s 2019-12-09T10:08:53.383 INFO:teuthology.orchestra.run.smithi059.stderr: 44950K .......... .......... .......... .......... .......... 61% 175M 0s 2019-12-09T10:08:53.383 INFO:teuthology.orchestra.run.smithi059.stderr: 45000K .......... .......... .......... .......... .......... 61% 231M 0s 2019-12-09T10:08:53.383 INFO:teuthology.orchestra.run.smithi059.stderr: 45050K .......... .......... .......... .......... .......... 61% 223M 0s 2019-12-09T10:08:53.383 INFO:teuthology.orchestra.run.smithi059.stderr: 45100K .......... .......... .......... .......... .......... 61% 251M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45150K .......... .......... .......... .......... .......... 61% 183M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45200K .......... .......... .......... .......... .......... 61% 233M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45250K .......... .......... .......... .......... .......... 61% 263M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45300K .......... .......... .......... .......... .......... 61% 191M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45350K .......... .......... .......... .......... .......... 61% 187M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45400K .......... .......... .......... .......... .......... 61% 214M 0s 2019-12-09T10:08:53.384 INFO:teuthology.orchestra.run.smithi059.stderr: 45450K .......... .......... .......... .......... .......... 61% 257M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45500K .......... .......... .......... .......... .......... 61% 278M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45550K .......... .......... .......... .......... .......... 61% 172M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45600K .......... .......... .......... .......... .......... 62% 201M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45650K .......... .......... .......... .......... .......... 62% 259M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45700K .......... .......... .......... .......... .......... 62% 181M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45750K .......... .......... .......... .......... .......... 62% 170M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45800K .......... .......... .......... .......... .......... 62% 241M 0s 2019-12-09T10:08:53.385 INFO:teuthology.orchestra.run.smithi059.stderr: 45850K .......... .......... .......... .......... .......... 62% 213M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 45900K .......... .......... .......... .......... .......... 62% 175M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 45950K .......... .......... .......... .......... .......... 62% 174M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 46000K .......... .......... .......... .......... .......... 62% 223M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 46050K .......... .......... .......... .......... .......... 62% 265M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 46100K .......... .......... .......... .......... .......... 62% 227M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 46150K .......... .......... .......... .......... .......... 62% 188M 0s 2019-12-09T10:08:53.386 INFO:teuthology.orchestra.run.smithi059.stderr: 46200K .......... .......... .......... .......... .......... 62% 214M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46250K .......... .......... .......... .......... .......... 62% 228M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46300K .......... .......... .......... .......... .......... 63% 249M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46350K .......... .......... .......... .......... .......... 63% 186M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46400K .......... .......... .......... .......... .......... 63% 261M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46450K .......... .......... .......... .......... .......... 63% 202M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46500K .......... .......... .......... .......... .......... 63% 239M 0s 2019-12-09T10:08:53.387 INFO:teuthology.orchestra.run.smithi059.stderr: 46550K .......... .......... .......... .......... .......... 63% 255M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46600K .......... .......... .......... .......... .......... 63% 258M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46650K .......... .......... .......... .......... .......... 63% 234M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46700K .......... .......... .......... .......... .......... 63% 245M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46750K .......... .......... .......... .......... .......... 63% 185M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46800K .......... .......... .......... .......... .......... 63% 254M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46850K .......... .......... .......... .......... .......... 63% 236M 0s 2019-12-09T10:08:53.388 INFO:teuthology.orchestra.run.smithi059.stderr: 46900K .......... .......... .......... .......... .......... 63% 267M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 46950K .......... .......... .......... .......... .......... 63% 188M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 47000K .......... .......... .......... .......... .......... 63% 227M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 47050K .......... .......... .......... .......... .......... 64% 244M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 47100K .......... .......... .......... .......... .......... 64% 254M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 47150K .......... .......... .......... .......... .......... 64% 252M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 47200K .......... .......... .......... .......... .......... 64% 245M 0s 2019-12-09T10:08:53.389 INFO:teuthology.orchestra.run.smithi059.stderr: 47250K .......... .......... .......... .......... .......... 64% 226M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47300K .......... .......... .......... .......... .......... 64% 225M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47350K .......... .......... .......... .......... .......... 64% 229M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47400K .......... .......... .......... .......... .......... 64% 233M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47450K .......... .......... .......... .......... .......... 64% 222M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47500K .......... .......... .......... .......... .......... 64% 281M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47550K .......... .......... .......... .......... .......... 64% 201M 0s 2019-12-09T10:08:53.390 INFO:teuthology.orchestra.run.smithi059.stderr: 47600K .......... .......... .......... .......... .......... 64% 175M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47650K .......... .......... .......... .......... .......... 64% 183M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47700K .......... .......... .......... .......... .......... 64% 229M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47750K .......... .......... .......... .......... .......... 64% 225M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47800K .......... .......... .......... .......... .......... 65% 221M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47850K .......... .......... .......... .......... .......... 65% 215M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47900K .......... .......... .......... .......... .......... 65% 218M 0s 2019-12-09T10:08:53.391 INFO:teuthology.orchestra.run.smithi059.stderr: 47950K .......... .......... .......... .......... .......... 65% 212M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48000K .......... .......... .......... .......... .......... 65% 256M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48050K .......... .......... .......... .......... .......... 65% 243M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48100K .......... .......... .......... .......... .......... 65% 271M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48150K .......... .......... .......... .......... .......... 65% 188M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48200K .......... .......... .......... .......... .......... 65% 235M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48250K .......... .......... .......... .......... .......... 65% 233M 0s 2019-12-09T10:08:53.392 INFO:teuthology.orchestra.run.smithi059.stderr: 48300K .......... .......... .......... .......... .......... 65% 257M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48350K .......... .......... .......... .......... .......... 65% 191M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48400K .......... .......... .......... .......... .......... 65% 258M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48450K .......... .......... .......... .......... .......... 65% 242M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48500K .......... .......... .......... .......... .......... 66% 222M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48550K .......... .......... .......... .......... .......... 66% 223M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48600K .......... .......... .......... .......... .......... 66% 216M 0s 2019-12-09T10:08:53.393 INFO:teuthology.orchestra.run.smithi059.stderr: 48650K .......... .......... .......... .......... .......... 66% 249M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 48700K .......... .......... .......... .......... .......... 66% 229M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 48750K .......... .......... .......... .......... .......... 66% 194M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 48800K .......... .......... .......... .......... .......... 66% 204M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 48850K .......... .......... .......... .......... .......... 66% 274M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 48900K .......... .......... .......... .......... .......... 66% 221M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 48950K .......... .......... .......... .......... .......... 66% 190M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 49000K .......... .......... .......... .......... .......... 66% 223M 0s 2019-12-09T10:08:53.394 INFO:teuthology.orchestra.run.smithi059.stderr: 49050K .......... .......... .......... .......... .......... 66% 269M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49100K .......... .......... .......... .......... .......... 66% 200M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49150K .......... .......... .......... .......... .......... 66% 225M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49200K .......... .......... .......... .......... .......... 66% 245M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49250K .......... .......... .......... .......... .......... 67% 206M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49300K .......... .......... .......... .......... .......... 67% 215M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49350K .......... .......... .......... .......... .......... 67% 183M 0s 2019-12-09T10:08:53.395 INFO:teuthology.orchestra.run.smithi059.stderr: 49400K .......... .......... .......... .......... .......... 67% 222M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49450K .......... .......... .......... .......... .......... 67% 243M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49500K .......... .......... .......... .......... .......... 67% 193M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49550K .......... .......... .......... .......... .......... 67% 209M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49600K .......... .......... .......... .......... .......... 67% 215M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49650K .......... .......... .......... .......... .......... 67% 251M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49700K .......... .......... .......... .......... .......... 67% 223M 0s 2019-12-09T10:08:53.396 INFO:teuthology.orchestra.run.smithi059.stderr: 49750K .......... .......... .......... .......... .......... 67% 210M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 49800K .......... .......... .......... .......... .......... 67% 208M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 49850K .......... .......... .......... .......... .......... 67% 228M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 49900K .......... .......... .......... .......... .......... 67% 222M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 49950K .......... .......... .......... .......... .......... 67% 202M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 50000K .......... .......... .......... .......... .......... 68% 257M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 50050K .......... .......... .......... .......... .......... 68% 245M 0s 2019-12-09T10:08:53.397 INFO:teuthology.orchestra.run.smithi059.stderr: 50100K .......... .......... .......... .......... .......... 68% 227M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50150K .......... .......... .......... .......... .......... 68% 182M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50200K .......... .......... .......... .......... .......... 68% 260M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50250K .......... .......... .......... .......... .......... 68% 267M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50300K .......... .......... .......... .......... .......... 68% 189M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50350K .......... .......... .......... .......... .......... 68% 181M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50400K .......... .......... .......... .......... .......... 68% 224M 0s 2019-12-09T10:08:53.398 INFO:teuthology.orchestra.run.smithi059.stderr: 50450K .......... .......... .......... .......... .......... 68% 220M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50500K .......... .......... .......... .......... .......... 68% 272M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50550K .......... .......... .......... .......... .......... 68% 189M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50600K .......... .......... .......... .......... .......... 68% 209M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50650K .......... .......... .......... .......... .......... 68% 263M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50700K .......... .......... .......... .......... .......... 68% 272M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50750K .......... .......... .......... .......... .......... 69% 196M 0s 2019-12-09T10:08:53.399 INFO:teuthology.orchestra.run.smithi059.stderr: 50800K .......... .......... .......... .......... .......... 69% 241M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 50850K .......... .......... .......... .......... .......... 69% 234M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 50900K .......... .......... .......... .......... .......... 69% 234M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 50950K .......... .......... .......... .......... .......... 69% 208M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 51000K .......... .......... .......... .......... .......... 69% 248M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 51050K .......... .......... .......... .......... .......... 69% 215M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 51100K .......... .......... .......... .......... .......... 69% 250M 0s 2019-12-09T10:08:53.400 INFO:teuthology.orchestra.run.smithi059.stderr: 51150K .......... .......... .......... .......... .......... 69% 221M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51200K .......... .......... .......... .......... .......... 69% 203M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51250K .......... .......... .......... .......... .......... 69% 232M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51300K .......... .......... .......... .......... .......... 69% 215M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51350K .......... .......... .......... .......... .......... 69% 191M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51400K .......... .......... .......... .......... .......... 69% 238M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51450K .......... .......... .......... .......... .......... 70% 242M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51500K .......... .......... .......... .......... .......... 70% 237M 0s 2019-12-09T10:08:53.401 INFO:teuthology.orchestra.run.smithi059.stderr: 51550K .......... .......... .......... .......... .......... 70% 198M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51600K .......... .......... .......... .......... .......... 70% 290M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51650K .......... .......... .......... .......... .......... 70% 214M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51700K .......... .......... .......... .......... .......... 70% 268M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51750K .......... .......... .......... .......... .......... 70% 181M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51800K .......... .......... .......... .......... .......... 70% 206M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51850K .......... .......... .......... .......... .......... 70% 203M 0s 2019-12-09T10:08:53.402 INFO:teuthology.orchestra.run.smithi059.stderr: 51900K .......... .......... .......... .......... .......... 70% 221M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 51950K .......... .......... .......... .......... .......... 70% 208M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 52000K .......... .......... .......... .......... .......... 70% 201M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 52050K .......... .......... .......... .......... .......... 70% 213M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 52100K .......... .......... .......... .......... .......... 70% 293M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 52150K .......... .......... .......... .......... .......... 70% 199M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 52200K .......... .......... .......... .......... .......... 71% 234M 0s 2019-12-09T10:08:53.403 INFO:teuthology.orchestra.run.smithi059.stderr: 52250K .......... .......... .......... .......... .......... 71% 215M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52300K .......... .......... .......... .......... .......... 71% 227M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52350K .......... .......... .......... .......... .......... 71% 203M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52400K .......... .......... .......... .......... .......... 71% 291M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52450K .......... .......... .......... .......... .......... 71% 256M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52500K .......... .......... .......... .......... .......... 71% 210M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52550K .......... .......... .......... .......... .......... 71% 192M 0s 2019-12-09T10:08:53.404 INFO:teuthology.orchestra.run.smithi059.stderr: 52600K .......... .......... .......... .......... .......... 71% 242M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52650K .......... .......... .......... .......... .......... 71% 241M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52700K .......... .......... .......... .......... .......... 71% 273M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52750K .......... .......... .......... .......... .......... 71% 187M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52800K .......... .......... .......... .......... .......... 71% 283M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52850K .......... .......... .......... .......... .......... 71% 246M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52900K .......... .......... .......... .......... .......... 71% 228M 0s 2019-12-09T10:08:53.405 INFO:teuthology.orchestra.run.smithi059.stderr: 52950K .......... .......... .......... .......... .......... 72% 194M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53000K .......... .......... .......... .......... .......... 72% 277M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53050K .......... .......... .......... .......... .......... 72% 222M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53100K .......... .......... .......... .......... .......... 72% 218M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53150K .......... .......... .......... .......... .......... 72% 197M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53200K .......... .......... .......... .......... .......... 72% 199M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53250K .......... .......... .......... .......... .......... 72% 227M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53300K .......... .......... .......... .......... .......... 72% 233M 0s 2019-12-09T10:08:53.406 INFO:teuthology.orchestra.run.smithi059.stderr: 53350K .......... .......... .......... .......... .......... 72% 205M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53400K .......... .......... .......... .......... .......... 72% 270M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53450K .......... .......... .......... .......... .......... 72% 221M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53500K .......... .......... .......... .......... .......... 72% 209M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53550K .......... .......... .......... .......... .......... 72% 184M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53600K .......... .......... .......... .......... .......... 72% 223M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53650K .......... .......... .......... .......... .......... 73% 232M 0s 2019-12-09T10:08:53.407 INFO:teuthology.orchestra.run.smithi059.stderr: 53700K .......... .......... .......... .......... .......... 73% 232M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 53750K .......... .......... .......... .......... .......... 73% 203M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 53800K .......... .......... .......... .......... .......... 73% 273M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 53850K .......... .......... .......... .......... .......... 73% 197M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 53900K .......... .......... .......... .......... .......... 73% 229M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 53950K .......... .......... .......... .......... .......... 73% 181M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 54000K .......... .......... .......... .......... .......... 73% 197M 0s 2019-12-09T10:08:53.408 INFO:teuthology.orchestra.run.smithi059.stderr: 54050K .......... .......... .......... .......... .......... 73% 289M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54100K .......... .......... .......... .......... .......... 73% 297M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54150K .......... .......... .......... .......... .......... 73% 167M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54200K .......... .......... .......... .......... .......... 73% 226M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54250K .......... .......... .......... .......... .......... 73% 218M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54300K .......... .......... .......... .......... .......... 73% 192M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54350K .......... .......... .......... .......... .......... 73% 233M 0s 2019-12-09T10:08:53.409 INFO:teuthology.orchestra.run.smithi059.stderr: 54400K .......... .......... .......... .......... .......... 74% 209M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54450K .......... .......... .......... .......... .......... 74% 207M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54500K .......... .......... .......... .......... .......... 74% 222M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54550K .......... .......... .......... .......... .......... 74% 195M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54600K .......... .......... .......... .......... .......... 74% 222M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54650K .......... .......... .......... .......... .......... 74% 241M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54700K .......... .......... .......... .......... .......... 74% 241M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54750K .......... .......... .......... .......... .......... 74% 188M 0s 2019-12-09T10:08:53.410 INFO:teuthology.orchestra.run.smithi059.stderr: 54800K .......... .......... .......... .......... .......... 74% 213M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 54850K .......... .......... .......... .......... .......... 74% 226M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 54900K .......... .......... .......... .......... .......... 74% 193M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 54950K .......... .......... .......... .......... .......... 74% 193M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 55000K .......... .......... .......... .......... .......... 74% 278M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 55050K .......... .......... .......... .......... .......... 74% 214M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 55100K .......... .......... .......... .......... .......... 74% 236M 0s 2019-12-09T10:08:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: 55150K .......... .......... .......... .......... .......... 75% 205M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55200K .......... .......... .......... .......... .......... 75% 215M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55250K .......... .......... .......... .......... .......... 75% 265M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55300K .......... .......... .......... .......... .......... 75% 217M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55350K .......... .......... .......... .......... .......... 75% 201M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55400K .......... .......... .......... .......... .......... 75% 265M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55450K .......... .......... .......... .......... .......... 75% 291M 0s 2019-12-09T10:08:53.412 INFO:teuthology.orchestra.run.smithi059.stderr: 55500K .......... .......... .......... .......... .......... 75% 207M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55550K .......... .......... .......... .......... .......... 75% 184M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55600K .......... .......... .......... .......... .......... 75% 232M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55650K .......... .......... .......... .......... .......... 75% 252M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55700K .......... .......... .......... .......... .......... 75% 240M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55750K .......... .......... .......... .......... .......... 75% 249M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55800K .......... .......... .......... .......... .......... 75% 192M 0s 2019-12-09T10:08:53.413 INFO:teuthology.orchestra.run.smithi059.stderr: 55850K .......... .......... .......... .......... .......... 75% 251M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 55900K .......... .......... .......... .......... .......... 76% 244M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 55950K .......... .......... .......... .......... .......... 76% 192M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 56000K .......... .......... .......... .......... .......... 76% 246M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 56050K .......... .......... .......... .......... .......... 76% 246M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 56100K .......... .......... .......... .......... .......... 76% 215M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 56150K .......... .......... .......... .......... .......... 76% 228M 0s 2019-12-09T10:08:53.414 INFO:teuthology.orchestra.run.smithi059.stderr: 56200K .......... .......... .......... .......... .......... 76% 254M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56250K .......... .......... .......... .......... .......... 76% 195M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56300K .......... .......... .......... .......... .......... 76% 244M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56350K .......... .......... .......... .......... .......... 76% 220M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56400K .......... .......... .......... .......... .......... 76% 213M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56450K .......... .......... .......... .......... .......... 76% 220M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56500K .......... .......... .......... .......... .......... 76% 225M 0s 2019-12-09T10:08:53.415 INFO:teuthology.orchestra.run.smithi059.stderr: 56550K .......... .......... .......... .......... .......... 76% 223M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56600K .......... .......... .......... .......... .......... 77% 231M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56650K .......... .......... .......... .......... .......... 77% 225M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56700K .......... .......... .......... .......... .......... 77% 278M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56750K .......... .......... .......... .......... .......... 77% 182M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56800K .......... .......... .......... .......... .......... 77% 196M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56850K .......... .......... .......... .......... .......... 77% 244M 0s 2019-12-09T10:08:53.416 INFO:teuthology.orchestra.run.smithi059.stderr: 56900K .......... .......... .......... .......... .......... 77% 168M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 56950K .......... .......... .......... .......... .......... 77% 194M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 57000K .......... .......... .......... .......... .......... 77% 228M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 57050K .......... .......... .......... .......... .......... 77% 219M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 57100K .......... .......... .......... .......... .......... 77% 217M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 57150K .......... .......... .......... .......... .......... 77% 195M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 57200K .......... .......... .......... .......... .......... 77% 217M 0s 2019-12-09T10:08:53.417 INFO:teuthology.orchestra.run.smithi059.stderr: 57250K .......... .......... .......... .......... .......... 77% 232M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57300K .......... .......... .......... .......... .......... 77% 215M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57350K .......... .......... .......... .......... .......... 78% 183M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57400K .......... .......... .......... .......... .......... 78% 269M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57450K .......... .......... .......... .......... .......... 78% 190M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57500K .......... .......... .......... .......... .......... 78% 223M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57550K .......... .......... .......... .......... .......... 78% 203M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57600K .......... .......... .......... .......... .......... 78% 218M 0s 2019-12-09T10:08:53.418 INFO:teuthology.orchestra.run.smithi059.stderr: 57650K .......... .......... .......... .......... .......... 78% 237M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 57700K .......... .......... .......... .......... .......... 78% 262M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 57750K .......... .......... .......... .......... .......... 78% 196M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 57800K .......... .......... .......... .......... .......... 78% 228M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 57850K .......... .......... .......... .......... .......... 78% 217M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 57900K .......... .......... .......... .......... .......... 78% 226M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 57950K .......... .......... .......... .......... .......... 78% 238M 0s 2019-12-09T10:08:53.419 INFO:teuthology.orchestra.run.smithi059.stderr: 58000K .......... .......... .......... .......... .......... 78% 279M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58050K .......... .......... .......... .......... .......... 78% 214M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58100K .......... .......... .......... .......... .......... 79% 347M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58150K .......... .......... .......... .......... .......... 79% 285M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58200K .......... .......... .......... .......... .......... 79% 405M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58250K .......... .......... .......... .......... .......... 79% 322M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58300K .......... .......... .......... .......... .......... 79% 385M 0s 2019-12-09T10:08:53.420 INFO:teuthology.orchestra.run.smithi059.stderr: 58350K .......... .......... .......... .......... .......... 79% 303M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58400K .......... .......... .......... .......... .......... 79% 314M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58450K .......... .......... .......... .......... .......... 79% 348M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58500K .......... .......... .......... .......... .......... 79% 409M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58550K .......... .......... .......... .......... .......... 79% 342M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58600K .......... .......... .......... .......... .......... 79% 370M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58650K .......... .......... .......... .......... .......... 79% 323M 0s 2019-12-09T10:08:53.421 INFO:teuthology.orchestra.run.smithi059.stderr: 58700K .......... .......... .......... .......... .......... 79% 416M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 58750K .......... .......... .......... .......... .......... 79% 342M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 58800K .......... .......... .......... .......... .......... 80% 338M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 58850K .......... .......... .......... .......... .......... 80% 335M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 58900K .......... .......... .......... .......... .......... 80% 412M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 58950K .......... .......... .......... .......... .......... 80% 325M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 59000K .......... .......... .......... .......... .......... 80% 318M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 59050K .......... .......... .......... .......... .......... 80% 357M 0s 2019-12-09T10:08:53.422 INFO:teuthology.orchestra.run.smithi059.stderr: 59100K .......... .......... .......... .......... .......... 80% 305M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59150K .......... .......... .......... .......... .......... 80% 280M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59200K .......... .......... .......... .......... .......... 80% 344M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59250K .......... .......... .......... .......... .......... 80% 358M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59300K .......... .......... .......... .......... .......... 80% 400M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59350K .......... .......... .......... .......... .......... 80% 357M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59400K .......... .......... .......... .......... .......... 80% 291M 0s 2019-12-09T10:08:53.423 INFO:teuthology.orchestra.run.smithi059.stderr: 59450K .......... .......... .......... .......... .......... 80% 358M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59500K .......... .......... .......... .......... .......... 80% 313M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59550K .......... .......... .......... .......... .......... 81% 267M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59600K .......... .......... .......... .......... .......... 81% 388M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59650K .......... .......... .......... .......... .......... 81% 398M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59700K .......... .......... .......... .......... .......... 81% 306M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59750K .......... .......... .......... .......... .......... 81% 285M 0s 2019-12-09T10:08:53.424 INFO:teuthology.orchestra.run.smithi059.stderr: 59800K .......... .......... .......... .......... .......... 81% 322M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 59850K .......... .......... .......... .......... .......... 81% 269M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 59900K .......... .......... .......... .......... .......... 81% 379M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 59950K .......... .......... .......... .......... .......... 81% 350M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 60000K .......... .......... .......... .......... .......... 81% 321M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 60050K .......... .......... .......... .......... .......... 81% 332M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 60100K .......... .......... .......... .......... .......... 81% 304M 0s 2019-12-09T10:08:53.425 INFO:teuthology.orchestra.run.smithi059.stderr: 60150K .......... .......... .......... .......... .......... 81% 343M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60200K .......... .......... .......... .......... .......... 81% 306M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60250K .......... .......... .......... .......... .......... 81% 338M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60300K .......... .......... .......... .......... .......... 82% 358M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60350K .......... .......... .......... .......... .......... 82% 303M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60400K .......... .......... .......... .......... .......... 82% 342M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60450K .......... .......... .......... .......... .......... 82% 394M 0s 2019-12-09T10:08:53.426 INFO:teuthology.orchestra.run.smithi059.stderr: 60500K .......... .......... .......... .......... .......... 82% 418M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60550K .......... .......... .......... .......... .......... 82% 292M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60600K .......... .......... .......... .......... .......... 82% 338M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60650K .......... .......... .......... .......... .......... 82% 337M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60700K .......... .......... .......... .......... .......... 82% 363M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60750K .......... .......... .......... .......... .......... 82% 315M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60800K .......... .......... .......... .......... .......... 82% 310M 0s 2019-12-09T10:08:53.427 INFO:teuthology.orchestra.run.smithi059.stderr: 60850K .......... .......... .......... .......... .......... 82% 326M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 60900K .......... .......... .......... .......... .......... 82% 419M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 60950K .......... .......... .......... .......... .......... 82% 299M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 61000K .......... .......... .......... .......... .......... 82% 383M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 61050K .......... .......... .......... .......... .......... 83% 366M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 61100K .......... .......... .......... .......... .......... 83% 392M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 61150K .......... .......... .......... .......... .......... 83% 264M 0s 2019-12-09T10:08:53.428 INFO:teuthology.orchestra.run.smithi059.stderr: 61200K .......... .......... .......... .......... .......... 83% 358M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61250K .......... .......... .......... .......... .......... 83% 365M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61300K .......... .......... .......... .......... .......... 83% 314M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61350K .......... .......... .......... .......... .......... 83% 292M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61400K .......... .......... .......... .......... .......... 83% 352M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61450K .......... .......... .......... .......... .......... 83% 319M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61500K .......... .......... .......... .......... .......... 83% 400M 0s 2019-12-09T10:08:53.429 INFO:teuthology.orchestra.run.smithi059.stderr: 61550K .......... .......... .......... .......... .......... 83% 258M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61600K .......... .......... .......... .......... .......... 83% 309M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61650K .......... .......... .......... .......... .......... 83% 316M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61700K .......... .......... .......... .......... .......... 83% 365M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61750K .......... .......... .......... .......... .......... 84% 278M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61800K .......... .......... .......... .......... .......... 84% 329M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61850K .......... .......... .......... .......... .......... 84% 318M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61900K .......... .......... .......... .......... .......... 84% 369M 0s 2019-12-09T10:08:53.430 INFO:teuthology.orchestra.run.smithi059.stderr: 61950K .......... .......... .......... .......... .......... 84% 310M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62000K .......... .......... .......... .......... .......... 84% 289M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62050K .......... .......... .......... .......... .......... 84% 397M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62100K .......... .......... .......... .......... .......... 84% 306M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62150K .......... .......... .......... .......... .......... 84% 271M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62200K .......... .......... .......... .......... .......... 84% 353M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62250K .......... .......... .......... .......... .......... 84% 349M 0s 2019-12-09T10:08:53.431 INFO:teuthology.orchestra.run.smithi059.stderr: 62300K .......... .......... .......... .......... .......... 84% 288M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62350K .......... .......... .......... .......... .......... 84% 292M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62400K .......... .......... .......... .......... .......... 84% 392M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62450K .......... .......... .......... .......... .......... 84% 338M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62500K .......... .......... .......... .......... .......... 85% 335M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62550K .......... .......... .......... .......... .......... 85% 305M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62600K .......... .......... .......... .......... .......... 85% 363M 0s 2019-12-09T10:08:53.432 INFO:teuthology.orchestra.run.smithi059.stderr: 62650K .......... .......... .......... .......... .......... 85% 337M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 62700K .......... .......... .......... .......... .......... 85% 367M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 62750K .......... .......... .......... .......... .......... 85% 349M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 62800K .......... .......... .......... .......... .......... 85% 348M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 62850K .......... .......... .......... .......... .......... 85% 397M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 62900K .......... .......... .......... .......... .......... 85% 321M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 62950K .......... .......... .......... .......... .......... 85% 282M 0s 2019-12-09T10:08:53.433 INFO:teuthology.orchestra.run.smithi059.stderr: 63000K .......... .......... .......... .......... .......... 85% 373M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63050K .......... .......... .......... .......... .......... 85% 367M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63100K .......... .......... .......... .......... .......... 85% 355M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63150K .......... .......... .......... .......... .......... 85% 361M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63200K .......... .......... .......... .......... .......... 85% 370M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63250K .......... .......... .......... .......... .......... 86% 411M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63300K .......... .......... .......... .......... .......... 86% 383M 0s 2019-12-09T10:08:53.434 INFO:teuthology.orchestra.run.smithi059.stderr: 63350K .......... .......... .......... .......... .......... 86% 303M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63400K .......... .......... .......... .......... .......... 86% 365M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63450K .......... .......... .......... .......... .......... 86% 394M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63500K .......... .......... .......... .......... .......... 86% 432M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63550K .......... .......... .......... .......... .......... 86% 319M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63600K .......... .......... .......... .......... .......... 86% 445M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63650K .......... .......... .......... .......... .......... 86% 387M 0s 2019-12-09T10:08:53.435 INFO:teuthology.orchestra.run.smithi059.stderr: 63700K .......... .......... .......... .......... .......... 86% 257M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 63750K .......... .......... .......... .......... .......... 86% 309M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 63800K .......... .......... .......... .......... .......... 86% 334M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 63850K .......... .......... .......... .......... .......... 86% 208M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 63900K .......... .......... .......... .......... .......... 86% 202M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 63950K .......... .......... .......... .......... .......... 87% 142M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 64000K .......... .......... .......... .......... .......... 87% 183M 0s 2019-12-09T10:08:53.436 INFO:teuthology.orchestra.run.smithi059.stderr: 64050K .......... .......... .......... .......... .......... 87% 177M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64100K .......... .......... .......... .......... .......... 87% 194M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64150K .......... .......... .......... .......... .......... 87% 193M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64200K .......... .......... .......... .......... .......... 87% 215M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64250K .......... .......... .......... .......... .......... 87% 284M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64300K .......... .......... .......... .......... .......... 87% 378M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64350K .......... .......... .......... .......... .......... 87% 301M 0s 2019-12-09T10:08:53.437 INFO:teuthology.orchestra.run.smithi059.stderr: 64400K .......... .......... .......... .......... .......... 87% 342M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64450K .......... .......... .......... .......... .......... 87% 370M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64500K .......... .......... .......... .......... .......... 87% 341M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64550K .......... .......... .......... .......... .......... 87% 306M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64600K .......... .......... .......... .......... .......... 87% 393M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64650K .......... .......... .......... .......... .......... 87% 340M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64700K .......... .......... .......... .......... .......... 88% 234M 0s 2019-12-09T10:08:53.438 INFO:teuthology.orchestra.run.smithi059.stderr: 64750K .......... .......... .......... .......... .......... 88% 329M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 64800K .......... .......... .......... .......... .......... 88% 348M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 64850K .......... .......... .......... .......... .......... 88% 407M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 64900K .......... .......... .......... .......... .......... 88% 356M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 64950K .......... .......... .......... .......... .......... 88% 301M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 65000K .......... .......... .......... .......... .......... 88% 368M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 65050K .......... .......... .......... .......... .......... 88% 355M 0s 2019-12-09T10:08:53.439 INFO:teuthology.orchestra.run.smithi059.stderr: 65100K .......... .......... .......... .......... .......... 88% 436M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65150K .......... .......... .......... .......... .......... 88% 315M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65200K .......... .......... .......... .......... .......... 88% 316M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65250K .......... .......... .......... .......... .......... 88% 405M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65300K .......... .......... .......... .......... .......... 88% 429M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65350K .......... .......... .......... .......... .......... 88% 239M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65400K .......... .......... .......... .......... .......... 88% 342M 0s 2019-12-09T10:08:53.440 INFO:teuthology.orchestra.run.smithi059.stderr: 65450K .......... .......... .......... .......... .......... 89% 326M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65500K .......... .......... .......... .......... .......... 89% 350M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65550K .......... .......... .......... .......... .......... 89% 260M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65600K .......... .......... .......... .......... .......... 89% 295M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65650K .......... .......... .......... .......... .......... 89% 358M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65700K .......... .......... .......... .......... .......... 89% 334M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65750K .......... .......... .......... .......... .......... 89% 306M 0s 2019-12-09T10:08:53.441 INFO:teuthology.orchestra.run.smithi059.stderr: 65800K .......... .......... .......... .......... .......... 89% 352M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 65850K .......... .......... .......... .......... .......... 89% 342M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 65900K .......... .......... .......... .......... .......... 89% 325M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 65950K .......... .......... .......... .......... .......... 89% 336M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 66000K .......... .......... .......... .......... .......... 89% 370M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 66050K .......... .......... .......... .......... .......... 89% 308M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 66100K .......... .......... .......... .......... .......... 89% 299M 0s 2019-12-09T10:08:53.442 INFO:teuthology.orchestra.run.smithi059.stderr: 66150K .......... .......... .......... .......... .......... 89% 343M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66200K .......... .......... .......... .......... .......... 90% 157M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66250K .......... .......... .......... .......... .......... 90% 230M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66300K .......... .......... .......... .......... .......... 90% 350M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66350K .......... .......... .......... .......... .......... 90% 193M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66400K .......... .......... .......... .......... .......... 90% 403M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66450K .......... .......... .......... .......... .......... 90% 287M 0s 2019-12-09T10:08:53.443 INFO:teuthology.orchestra.run.smithi059.stderr: 66500K .......... .......... .......... .......... .......... 90% 320M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66550K .......... .......... .......... .......... .......... 90% 300M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66600K .......... .......... .......... .......... .......... 90% 349M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66650K .......... .......... .......... .......... .......... 90% 345M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66700K .......... .......... .......... .......... .......... 90% 298M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66750K .......... .......... .......... .......... .......... 90% 300M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66800K .......... .......... .......... .......... .......... 90% 344M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66850K .......... .......... .......... .......... .......... 90% 292M 0s 2019-12-09T10:08:53.444 INFO:teuthology.orchestra.run.smithi059.stderr: 66900K .......... .......... .......... .......... .......... 91% 394M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 66950K .......... .......... .......... .......... .......... 91% 325M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 67000K .......... .......... .......... .......... .......... 91% 170M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 67050K .......... .......... .......... .......... .......... 91% 367M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 67100K .......... .......... .......... .......... .......... 91% 362M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 67150K .......... .......... .......... .......... .......... 91% 318M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 67200K .......... .......... .......... .......... .......... 91% 380M 0s 2019-12-09T10:08:53.445 INFO:teuthology.orchestra.run.smithi059.stderr: 67250K .......... .......... .......... .......... .......... 91% 308M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67300K .......... .......... .......... .......... .......... 91% 269M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67350K .......... .......... .......... .......... .......... 91% 317M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67400K .......... .......... .......... .......... .......... 91% 340M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67450K .......... .......... .......... .......... .......... 91% 292M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67500K .......... .......... .......... .......... .......... 91% 269M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67550K .......... .......... .......... .......... .......... 91% 186M 0s 2019-12-09T10:08:53.446 INFO:teuthology.orchestra.run.smithi059.stderr: 67600K .......... .......... .......... .......... .......... 91% 293M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67650K .......... .......... .......... .......... .......... 92% 382M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67700K .......... .......... .......... .......... .......... 92% 354M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67750K .......... .......... .......... .......... .......... 92% 177M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67800K .......... .......... .......... .......... .......... 92% 334M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67850K .......... .......... .......... .......... .......... 92% 326M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67900K .......... .......... .......... .......... .......... 92% 221M 0s 2019-12-09T10:08:53.447 INFO:teuthology.orchestra.run.smithi059.stderr: 67950K .......... .......... .......... .......... .......... 92% 318M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68000K .......... .......... .......... .......... .......... 92% 339M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68050K .......... .......... .......... .......... .......... 92% 324M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68100K .......... .......... .......... .......... .......... 92% 294M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68150K .......... .......... .......... .......... .......... 92% 309M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68200K .......... .......... .......... .......... .......... 92% 348M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68250K .......... .......... .......... .......... .......... 92% 284M 0s 2019-12-09T10:08:53.448 INFO:teuthology.orchestra.run.smithi059.stderr: 68300K .......... .......... .......... .......... .......... 92% 314M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68350K .......... .......... .......... .......... .......... 92% 258M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68400K .......... .......... .......... .......... .......... 93% 280M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68450K .......... .......... .......... .......... .......... 93% 177M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68500K .......... .......... .......... .......... .......... 93% 375M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68550K .......... .......... .......... .......... .......... 93% 311M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68600K .......... .......... .......... .......... .......... 93% 317M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68650K .......... .......... .......... .......... .......... 93% 334M 0s 2019-12-09T10:08:53.449 INFO:teuthology.orchestra.run.smithi059.stderr: 68700K .......... .......... .......... .......... .......... 93% 361M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 68750K .......... .......... .......... .......... .......... 93% 271M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 68800K .......... .......... .......... .......... .......... 93% 226M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 68850K .......... .......... .......... .......... .......... 93% 218M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 68900K .......... .......... .......... .......... .......... 93% 309M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 68950K .......... .......... .......... .......... .......... 93% 323M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 69000K .......... .......... .......... .......... .......... 93% 341M 0s 2019-12-09T10:08:53.450 INFO:teuthology.orchestra.run.smithi059.stderr: 69050K .......... .......... .......... .......... .......... 93% 339M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69100K .......... .......... .......... .......... .......... 94% 409M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69150K .......... .......... .......... .......... .......... 94% 301M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69200K .......... .......... .......... .......... .......... 94% 346M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69250K .......... .......... .......... .......... .......... 94% 213M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69300K .......... .......... .......... .......... .......... 94% 259M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69350K .......... .......... .......... .......... .......... 94% 321M 0s 2019-12-09T10:08:53.451 INFO:teuthology.orchestra.run.smithi059.stderr: 69400K .......... .......... .......... .......... .......... 94% 335M 0s 2019-12-09T10:08:53.452 INFO:teuthology.orchestra.run.smithi059.stderr: 69450K .......... .......... .......... .......... .......... 94% 281M 0s 2019-12-09T10:08:53.452 INFO:teuthology.orchestra.run.smithi059.stderr: 69500K .......... .......... .......... .......... .......... 94% 269M 0s 2019-12-09T10:08:53.452 INFO:teuthology.orchestra.run.smithi059.stderr: 69550K .......... .......... .......... .......... .......... 94% 217M 0s 2019-12-09T10:08:53.452 INFO:teuthology.orchestra.run.smithi059.stderr: 69600K .......... .......... .......... .......... .......... 94% 381M 0s 2019-12-09T10:08:53.452 INFO:teuthology.orchestra.run.smithi059.stderr: 69650K .......... .......... .......... .......... .......... 94% 372M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 69700K .......... .......... .......... .......... .......... 94% 280M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 69750K .......... .......... .......... .......... .......... 94% 249M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 69800K .......... .......... .......... .......... .......... 94% 386M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 69850K .......... .......... .......... .......... .......... 95% 324M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 69900K .......... .......... .......... .......... .......... 95% 255M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 69950K .......... .......... .......... .......... .......... 95% 186M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 70000K .......... .......... .......... .......... .......... 95% 109M 0s 2019-12-09T10:08:53.453 INFO:teuthology.orchestra.run.smithi059.stderr: 70050K .......... .......... .......... .......... .......... 95% 131M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70100K .......... .......... .......... .......... .......... 95% 147M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70150K .......... .......... .......... .......... .......... 95% 163M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70200K .......... .......... .......... .......... .......... 95% 147M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70250K .......... .......... .......... .......... .......... 95% 183M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70300K .......... .......... .......... .......... .......... 95% 183M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70350K .......... .......... .......... .......... .......... 95% 148M 0s 2019-12-09T10:08:53.454 INFO:teuthology.orchestra.run.smithi059.stderr: 70400K .......... .......... .......... .......... .......... 95% 183M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70450K .......... .......... .......... .......... .......... 95% 178M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70500K .......... .......... .......... .......... .......... 95% 176M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70550K .......... .......... .......... .......... .......... 95% 157M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70600K .......... .......... .......... .......... .......... 96% 187M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70650K .......... .......... .......... .......... .......... 96% 172M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70700K .......... .......... .......... .......... .......... 96% 176M 0s 2019-12-09T10:08:53.455 INFO:teuthology.orchestra.run.smithi059.stderr: 70750K .......... .......... .......... .......... .......... 96% 138M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 70800K .......... .......... .......... .......... .......... 96% 175M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 70850K .......... .......... .......... .......... .......... 96% 170M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 70900K .......... .......... .......... .......... .......... 96% 188M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 70950K .......... .......... .......... .......... .......... 96% 157M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 71000K .......... .......... .......... .......... .......... 96% 178M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 71050K .......... .......... .......... .......... .......... 96% 173M 0s 2019-12-09T10:08:53.456 INFO:teuthology.orchestra.run.smithi059.stderr: 71100K .......... .......... .......... .......... .......... 96% 178M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71150K .......... .......... .......... .......... .......... 96% 150M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71200K .......... .......... .......... .......... .......... 96% 184M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71250K .......... .......... .......... .......... .......... 96% 155M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71300K .......... .......... .......... .......... .......... 96% 169M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71350K .......... .......... .......... .......... .......... 97% 143M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71400K .......... .......... .......... .......... .......... 97% 156M 0s 2019-12-09T10:08:53.457 INFO:teuthology.orchestra.run.smithi059.stderr: 71450K .......... .......... .......... .......... .......... 97% 170M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71500K .......... .......... .......... .......... .......... 97% 171M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71550K .......... .......... .......... .......... .......... 97% 148M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71600K .......... .......... .......... .......... .......... 97% 189M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71650K .......... .......... .......... .......... .......... 97% 173M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71700K .......... .......... .......... .......... .......... 97% 232M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71750K .......... .......... .......... .......... .......... 97% 197M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71800K .......... .......... .......... .......... .......... 97% 222M 0s 2019-12-09T10:08:53.458 INFO:teuthology.orchestra.run.smithi059.stderr: 71850K .......... .......... .......... .......... .......... 97% 259M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 71900K .......... .......... .......... .......... .......... 97% 249M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 71950K .......... .......... .......... .......... .......... 97% 160M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 72000K .......... .......... .......... .......... .......... 97% 205M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 72050K .......... .......... .......... .......... .......... 98% 217M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 72100K .......... .......... .......... .......... .......... 98% 272M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 72150K .......... .......... .......... .......... .......... 98% 227M 0s 2019-12-09T10:08:53.459 INFO:teuthology.orchestra.run.smithi059.stderr: 72200K .......... .......... .......... .......... .......... 98% 271M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72250K .......... .......... .......... .......... .......... 98% 276M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72300K .......... .......... .......... .......... .......... 98% 285M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72350K .......... .......... .......... .......... .......... 98% 234M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72400K .......... .......... .......... .......... .......... 98% 278M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72450K .......... .......... .......... .......... .......... 98% 269M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72500K .......... .......... .......... .......... .......... 98% 272M 0s 2019-12-09T10:08:53.460 INFO:teuthology.orchestra.run.smithi059.stderr: 72550K .......... .......... .......... .......... .......... 98% 249M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72600K .......... .......... .......... .......... .......... 98% 282M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72650K .......... .......... .......... .......... .......... 98% 279M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72700K .......... .......... .......... .......... .......... 98% 283M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72750K .......... .......... .......... .......... .......... 98% 231M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72800K .......... .......... .......... .......... .......... 99% 289M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72850K .......... .......... .......... .......... .......... 99% 269M 0s 2019-12-09T10:08:53.461 INFO:teuthology.orchestra.run.smithi059.stderr: 72900K .......... .......... .......... .......... .......... 99% 278M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 72950K .......... .......... .......... .......... .......... 99% 255M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 73000K .......... .......... .......... .......... .......... 99% 275M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 73050K .......... .......... .......... .......... .......... 99% 287M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 73100K .......... .......... .......... .......... .......... 99% 288M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 73150K .......... .......... .......... .......... .......... 99% 239M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 73200K .......... .......... .......... .......... .......... 99% 288M 0s 2019-12-09T10:08:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: 73250K .......... .......... .......... .......... .......... 99% 277M 0s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 73300K .......... .......... .......... .......... .......... 99% 283M 0s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 73350K .......... .......... .......... .......... .......... 99% 250M 0s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 73400K .......... .......... .......... .......... .......... 99% 276M 0s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 73450K .......... .......... .......... .......... .......... 99% 281M 0s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 73500K .......... .......... .......... .......... .......... 99% 284M 0s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 73550K ........ 100% 147M=0.4s 2019-12-09T10:08:53.463 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T10:08:53.464 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09 10:08:52 (197 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-12-09T10:08:53.464 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T10:08:53.464 INFO:teuthology.orchestra.run.smithi059.stdout:Archive: gradle-4.7-bin.zip 2019-12-09T10:08:53.464 INFO:teuthology.orchestra.run.smithi059.stdout: creating: /opt/gradle/gradle-4.7/ 2019-12-09T10:08:53.464 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-12-09T10:08:53.464 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-12-09T10:08:53.465 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-12-09T10:08:53.466 INFO:teuthology.orchestra.run.smithi059.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-12-09T10:08:53.467 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-12-09T10:08:53.468 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-12-09T10:08:53.469 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-12-09T10:08:53.470 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-12-09T10:08:53.471 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-12-09T10:08:53.472 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-12-09T10:08:53.473 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-12-09T10:08:53.474 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-12-09T10:08:53.475 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-12-09T10:08:53.476 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-12-09T10:08:53.477 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-12-09T10:08:53.478 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-12-09T10:08:53.479 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-12-09T10:08:53.480 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-12-09T10:08:53.481 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-12-09T10:08:53.481 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-12-09T10:08:53.481 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-12-09T10:08:53.481 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-12-09T10:08:53.481 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-12-09T10:08:53.481 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-12-09T10:08:53.482 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-12-09T10:08:53.483 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-12-09T10:08:53.484 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-12-09T10:08:53.484 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-12-09T10:08:53.484 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-12-09T10:08:53.484 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-12-09T10:08:53.484 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-12-09T10:08:53.485 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-12-09T10:08:53.486 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-12-09T10:08:53.487 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-12-09T10:08:53.488 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-12-09T10:08:53.489 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-12-09T10:08:53.490 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-12-09T10:08:53.491 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-12-09T10:08:53.492 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-12-09T10:08:53.493 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-12-09T10:08:53.494 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-12-09T10:08:53.495 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-12-09T10:08:53.495 INFO:teuthology.orchestra.run.smithi059.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-12-09T10:08:53.495 INFO:teuthology.orchestra.run.smithi059.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-12-09T10:08:54.322 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.322 INFO:teuthology.orchestra.run.smithi059.stdout:Welcome to Gradle 4.7! 2019-12-09T10:08:54.323 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.323 INFO:teuthology.orchestra.run.smithi059.stdout:Here are the highlights of this release: 2019-12-09T10:08:54.323 INFO:teuthology.orchestra.run.smithi059.stdout: - Incremental annotation processing 2019-12-09T10:08:54.323 INFO:teuthology.orchestra.run.smithi059.stdout: - JDK 10 support 2019-12-09T10:08:54.324 INFO:teuthology.orchestra.run.smithi059.stdout: - Grouped non-interactive console logs 2019-12-09T10:08:54.324 INFO:teuthology.orchestra.run.smithi059.stdout: - Failed tests are re-run first for quicker feedback 2019-12-09T10:08:54.324 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.324 INFO:teuthology.orchestra.run.smithi059.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-12-09T10:08:54.325 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.669 INFO:teuthology.orchestra.run.smithi059.stdout:------------------------------------------------------------ 2019-12-09T10:08:54.669 INFO:teuthology.orchestra.run.smithi059.stdout:Gradle 4.7 2019-12-09T10:08:54.669 INFO:teuthology.orchestra.run.smithi059.stdout:------------------------------------------------------------ 2019-12-09T10:08:54.669 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.669 INFO:teuthology.orchestra.run.smithi059.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout:Groovy: 2.4.12 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout:OS: Linux 3.10.0-1062.9.1.el7.x86_64 amd64 2019-12-09T10:08:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:54.671 INFO:teuthology.orchestra.run.smithi059:> sudo keytool -import -alias smithi059.front.sepia.ceph.com -keystore $(readlink -e $(dirname $(readlink -e $(which keytool)))/../lib/security/cacerts) -file /home/ubuntu/cephtest/ca/rgw.client.0.crt -storepass changeit 2019-12-09T10:08:55.087 INFO:teuthology.orchestra.run.smithi059.stderr:Certificate was added to keystore 2019-12-09T10:08:55.107 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-12-09T10:08:55.107 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T10:08:55.108 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi059.front.sepia.ceph.com 2019-12-09T10:08:55.108 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T10:08:55.108 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi161.front.sepia.ceph.com 2019-12-09T10:08:55.108 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-12-09T10:08:55.535 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-12-09T10:08:55.544 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-12-09T10:08:55.545 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'SYMSZSLQICTYGPOCWWGT', 's3dir': 'folder', 'endpoint': 'smithi059.front.sepia.ceph.com:443', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 443, 'access_secret': 'Z1y3MA6szI2HoIZrRpApEAC4gOG2Mzc6o9ZEe+a6/6s2xgncPpPlsA==', 'host': 'smithi059.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': True, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-12-09T10:08:55.545 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-12-09T10:08:55.545 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'SYMSZSLQICTYGPOCWWGT', '--secret', 'Z1y3MA6szI2HoIZrRpApEAC4gOG2Mzc6o9ZEe+a6/6s2xgncPpPlsA==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-12-09T10:08:55.546 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key SYMSZSLQICTYGPOCWWGT --secret Z1y3MA6szI2HoIZrRpApEAC4gOG2Mzc6o9ZEe+a6/6s2xgncPpPlsA== --email tester.client.0_test@test.test --cluster ceph 2019-12-09T10:08:55.700 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.706+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.725 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.731+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:55.725 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.731+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.739+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.739+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.739 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.745+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:55.739 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.745+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.761+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.962+0000 7fd3c34f6ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:08:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.962+0000 7fd3c34f6ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:08:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.962+0000 7fd3c34f6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:08:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.962+0000 7fd3c34f6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:08:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.962+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.974+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:55.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.975+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.987+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:08:55.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:55.987+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.001+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.001+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.013+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:08:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.013+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.019+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.019+0000 7fd3c34f6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:08:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.019+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.024+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.024+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.033+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.033+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.038 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.044+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:08:56.038 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.044+0000 7fd3c34f6ec0 20 zonegroup default 2019-12-09T10:08:56.038 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.044+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.050+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.050+0000 7fd3c34f6ec0 10 Cannot find current period zone using local zone 2019-12-09T10:08:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.050+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.055+0000 7fd3c34f6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:56.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.055+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.064+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:08:56.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.064+0000 7fd3c34f6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.073+0000 7fd3c34f6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:08:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.073+0000 7fd3c34f6ec0 20 zone default found 2019-12-09T10:08:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.073+0000 7fd3c34f6ec0 20 started sync module instance, tier type = 2019-12-09T10:08:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.073+0000 7fd3c34f6ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:08:56.401 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.406+0000 7fd3c34f6ec0 20 add_watcher() i=0 2019-12-09T10:08:56.401 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.406+0000 7fd3c34f6ec0 20 add_watcher() i=1 2019-12-09T10:08:56.401 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.406+0000 7fd3c34f6ec0 20 add_watcher() i=2 2019-12-09T10:08:56.401 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.406+0000 7fd3c34f6ec0 20 add_watcher() i=3 2019-12-09T10:08:56.401 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.406+0000 7fd3c34f6ec0 20 add_watcher() i=4 2019-12-09T10:08:56.401 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.406+0000 7fd3c34f6ec0 20 add_watcher() i=5 2019-12-09T10:08:56.405 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.411+0000 7fd3c34f6ec0 20 add_watcher() i=6 2019-12-09T10:08:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.435+0000 7fd3c34f6ec0 20 add_watcher() i=7 2019-12-09T10:08:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.435+0000 7fd3c34f6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:08:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.435+0000 7fd380fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:08:56.432 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.438+0000 7fd3b0720700 0 -- 172.21.15.59:0/1572086744 >> v2:172.21.15.161:3300/0 conn(0x5580f86c8d30 msgr2=0x5580f86cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:08:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:56.473+0000 7fd3aff1f700 0 -- 172.21.15.59:0/2973364813 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x5580f86e1e50 msgr2=0x5580f86e4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:08:58.428 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.433+0000 7fd2f2ffd700 20 reqs_thread_entry: start 2019-12-09T10:08:58.428 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.433+0000 7fd2f0ff9700 20 reqs_thread_entry: start 2019-12-09T10:08:58.428 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.433+0000 7fd3c34f6ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:08:58.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.434+0000 7fd2e6ffd700 20 reqs_thread_entry: start 2019-12-09T10:08:58.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.435+0000 7fd3c34f6ec0 20 get_system_obj_state: rctx=0x7ffefa91e4f8 obj=default.rgw.meta:users.uid:tester.client.0 state=0x5580f874b6d0 s->prefetch_data=0 2019-12-09T10:08:58.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.435+0000 7fd3c34f6ec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-12-09T10:08:58.437 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.442+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-12-09T10:08:58.437 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.442+0000 7fd3c34f6ec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T10:08:58.437 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.442+0000 7fd3c34f6ec0 20 get_system_obj_state: rctx=0x7ffefa91e508 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x5580f874d460 s->prefetch_data=0 2019-12-09T10:08:58.437 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.443+0000 7fd3c34f6ec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-12-09T10:08:58.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.451+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-12-09T10:08:58.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.451+0000 7fd3c34f6ec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T10:08:58.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.451+0000 7fd3c34f6ec0 20 get_system_obj_state: rctx=0x7ffefa91e508 obj=default.rgw.meta:users.keys:SYMSZSLQICTYGPOCWWGT state=0x5580f874d460 s->prefetch_data=0 2019-12-09T10:08:58.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.451+0000 7fd3c34f6ec0 10 cache get: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT : miss 2019-12-09T10:08:58.453 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.459+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT info.flags=0x0 2019-12-09T10:08:58.453 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.459+0000 7fd3c34f6ec0 10 adding default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT to cache LRU end 2019-12-09T10:08:58.453 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.459+0000 7fd3c34f6ec0 20 get_system_obj_state: rctx=0x7ffefa91e118 obj=default.rgw.meta:users.keys:SYMSZSLQICTYGPOCWWGT state=0x5580f874d460 s->prefetch_data=0 2019-12-09T10:08:58.453 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.459+0000 7fd3c34f6ec0 10 cache get: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT : type miss (requested=0x6, cached=0x0) 2019-12-09T10:08:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.465+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT info.flags=0x0 2019-12-09T10:08:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.465+0000 7fd3c34f6ec0 10 moving default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT to cache LRU end 2019-12-09T10:08:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.465+0000 7fd3c34f6ec0 20 get_system_obj_state: rctx=0x7ffefa91e298 obj=default.rgw.meta:users.keys:SYMSZSLQICTYGPOCWWGT state=0x5580f874d460 s->prefetch_data=0 2019-12-09T10:08:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.465+0000 7fd3c34f6ec0 10 cache get: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT : type miss (requested=0x6, cached=0x0) 2019-12-09T10:08:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.471+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT info.flags=0x0 2019-12-09T10:08:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.471+0000 7fd3c34f6ec0 10 moving default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT to cache LRU end 2019-12-09T10:08:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.521+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T10:08:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.521+0000 7fd3c34f6ec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T10:08:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.521+0000 7fd3c34f6ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-12-09T10:08:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.537+0000 7fd395ffb700 10 RGWWatcher::handle_notify() notify_id 584115552258 cookie 94012411985680 notifier 4505 bl.length()=632 2019-12-09T10:08:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.537+0000 7fd395ffb700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T10:08:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.537+0000 7fd395ffb700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T10:08:58.875 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.882+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T10:08:58.876 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.882+0000 7fd3c34f6ec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T10:08:58.876 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.882+0000 7fd3c34f6ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-12-09T10:08:58.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.894+0000 7fd395ffb700 10 RGWWatcher::handle_notify() notify_id 584115552259 cookie 94012411978464 notifier 4505 bl.length()=200 2019-12-09T10:08:58.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.894+0000 7fd395ffb700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T10:08:58.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:58.894+0000 7fd395ffb700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T10:08:58.924 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:25:55.552 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:58.974 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:25:55.602 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:58.989 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:25:55.618 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.006 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:25:55.635 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.070+0000 7fd3c34f6ec0 10 cache put: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT info.flags=0x7 2019-12-09T10:08:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.070+0000 7fd3c34f6ec0 10 moving default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT to cache LRU end 2019-12-09T10:08:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.070+0000 7fd3c34f6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=189 2019-12-09T10:08:59.065 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:25:55.669 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.068 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:25:55.673 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.098 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.105+0000 7fd395ffb700 10 RGWWatcher::handle_notify() notify_id 584115552269 cookie 94012411973008 notifier 4505 bl.length()=189 2019-12-09T10:08:59.098 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.105+0000 7fd395ffb700 10 cache put: name=default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT info.flags=0x7 2019-12-09T10:08:59.099 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.105+0000 7fd395ffb700 10 moving default.rgw.meta+users.keys+SYMSZSLQICTYGPOCWWGT to cache LRU end 2019-12-09T10:08:59.115 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:25:55.744 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.142 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:25:55.770 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.144 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T10:08:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "tester.client.0", 2019-12-09T10:08:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Ms. tester.client.0", 2019-12-09T10:08:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "tester.client.0_test@test.test", 2019-12-09T10:08:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T10:08:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T10:08:59.145 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [], 2019-12-09T10:08:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [ 2019-12-09T10:08:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T10:08:59.146 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "tester.client.0", 2019-12-09T10:08:59.149 INFO:teuthology.orchestra.run.smithi059.stdout: "access_key": "SYMSZSLQICTYGPOCWWGT", 2019-12-09T10:08:59.150 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "Z1y3MA6szI2HoIZrRpApEAC4gOG2Mzc6o9ZEe+a6/6s2xgncPpPlsA==" 2019-12-09T10:08:59.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T10:08:59.150 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T10:08:59.150 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [], 2019-12-09T10:08:59.150 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T10:08:59.150 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T10:08:59.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T10:08:59.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T10:08:59.151 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T10:08:59.151 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T10:08:59.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T10:08:59.152 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T10:08:59.153 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T10:08:59.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T10:08:59.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T10:08:59.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T10:08:59.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T10:08:59.154 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T10:08:59.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T10:08:59.160 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T10:08:59.160 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T10:08:59.161 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:08:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.188+0000 7fd3c34f6ec0 20 remove_watcher() i=0 2019-12-09T10:08:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.188+0000 7fd3c34f6ec0 2 removed watcher, disabling cache 2019-12-09T10:08:59.250 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.257+0000 7fd3c34f6ec0 20 remove_watcher() i=1 2019-12-09T10:08:59.254 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:25:55.860 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:08:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.299+0000 7fd3c34f6ec0 20 remove_watcher() i=2 2019-12-09T10:08:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.328+0000 7fd3c34f6ec0 20 remove_watcher() i=3 2019-12-09T10:08:59.356 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.363+0000 7fd3c34f6ec0 20 remove_watcher() i=4 2019-12-09T10:08:59.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.403+0000 7fd3c34f6ec0 20 remove_watcher() i=5 2019-12-09T10:08:59.431 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.438+0000 7fd3c34f6ec0 20 remove_watcher() i=6 2019-12-09T10:08:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.475+0000 7fd3c34f6ec0 20 remove_watcher() i=7 2019-12-09T10:08:59.481 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-12-09T10:08:59.542 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-12-09T10:08:59.544 INFO:teuthology.orchestra.run.smithi059:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-12-09T10:08:59.618 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:08:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.900+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:59.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.941+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:59.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.941+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:59.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.950+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:59.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.950+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:59.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.957+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:08:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.957+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:08:59.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:08:59.975+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:00.158 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.164+0000 7ffa5421bec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:09:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.164+0000 7ffa5421bec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:09:00.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.164+0000 7ffa5421bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:09:00.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.164+0000 7ffa5421bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:09:00.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.164+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.173+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.174+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.177 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.184+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:09:00.177 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.184+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.194+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.194+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.206+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:09:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.206+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.215+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.215+0000 7ffa5421bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:09:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.215+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.213 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.220+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:00.213 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.220+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.228+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:00.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.228+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.236+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:09:00.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.236+0000 7ffa5421bec0 20 zonegroup default 2019-12-09T10:09:00.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.236+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.242+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.242+0000 7ffa5421bec0 10 Cannot find current period zone using local zone 2019-12-09T10:09:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.242+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.247+0000 7ffa5421bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.247+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.256+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.256+0000 7ffa5421bec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:00.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.264+0000 7ffa5421bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:09:00.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.264+0000 7ffa5421bec0 20 zone default found 2019-12-09T10:09:00.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.264+0000 7ffa5421bec0 20 started sync module instance, tier type = 2019-12-09T10:09:00.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.264+0000 7ffa5421bec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:09:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.634+0000 7ffa5421bec0 20 add_watcher() i=0 2019-12-09T10:09:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.634+0000 7ffa5421bec0 20 add_watcher() i=1 2019-12-09T10:09:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.634+0000 7ffa5421bec0 20 add_watcher() i=2 2019-12-09T10:09:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.634+0000 7ffa5421bec0 20 add_watcher() i=3 2019-12-09T10:09:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.634+0000 7ffa5421bec0 20 add_watcher() i=4 2019-12-09T10:09:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.634+0000 7ffa5421bec0 20 add_watcher() i=5 2019-12-09T10:09:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.636+0000 7ffa5421bec0 20 add_watcher() i=6 2019-12-09T10:09:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.667+0000 7ffa5421bec0 20 add_watcher() i=7 2019-12-09T10:09:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.667+0000 7ffa5421bec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:09:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.667+0000 7ffa11fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:09:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.669+0000 7ffa3bfff700 0 -- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55769a98f6d0 msgr2=0x55769a98fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:09:00.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:00.726+0000 7ffa41445700 0 -- 172.21.15.59:0/1575813785 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7ff984022480 msgr2=0x7ff984024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:09:02.610 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.617+0000 7ff97bfff700 20 reqs_thread_entry: start 2019-12-09T10:09:02.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.618+0000 7ff979ffb700 20 reqs_thread_entry: start 2019-12-09T10:09:02.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.618+0000 7ffa5421bec0 20 init_complete bucket index max shards: 0 2019-12-09T10:09:02.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.618+0000 7ff96bfff700 20 reqs_thread_entry: start 2019-12-09T10:09:02.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.618+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:09:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.702+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.748+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:02.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:09:02.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_iEPmserpwZTTurzmB3Xta6cR_-osotZ', time=2019-12-09T12:08:28.486232+0000, chain.objs.size()=1 2019-12-09T10:09:02.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.AF2c5irormqXW3Lv-Tkt-ggn4GMkk9E_1 2019-12-09T10:09:02.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.112', time=2019-12-09T12:08:28.701221+0000, chain.objs.size()=2 2019-12-09T10:09:02.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.VvfoU1cD5kQD5gjAXheYN4lxF1IlEXu_0 2019-12-09T10:09:02.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.VvfoU1cD5kQD5gjAXheYN4lxF1IlEXu_1 2019-12-09T10:09:02.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.114', time=2019-12-09T12:08:29.133056+0000, chain.objs.size()=2 2019-12-09T10:09:02.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.VvfoU1cD5kQD5gjAXheYN4lxF1IlEXu_0 2019-12-09T10:09:02.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.811+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.VvfoU1cD5kQD5gjAXheYN4lxF1IlEXu_1 2019-12-09T10:09:02.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:02.969+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.079+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:09:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.119+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.140+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:03.143 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.150+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:03.178 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.184+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:09:03.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.236+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:03.255 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.261+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:03.293 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.299+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:03.293 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.299+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.242', time=2019-12-09T12:08:33.803009+0000, chain.objs.size()=3 2019-12-09T10:09:03.293 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.299+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-STANDARD.2~G2V5yRXQ5xi5miuX_5anXe9M3TeSPEw.1 2019-12-09T10:09:03.293 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.299+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-STANDARD.2~G2V5yRXQ5xi5miuX_5anXe9M3TeSPEw.2 2019-12-09T10:09:03.293 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.299+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-STANDARD.2~G2V5yRXQ5xi5miuX_5anXe9M3TeSPEw.3 2019-12-09T10:09:03.470 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.476+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:03.572 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.579+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:09:03.631 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.638+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:03.700 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.706+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:03.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.736+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:03.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.736+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_zwu9znG7QsMdQ-v_KAywQMR2wQEjenU', time=2019-12-09T12:08:28.958779+0000, chain.objs.size()=2 2019-12-09T10:09:03.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.736+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.nfO3hAg-QUaXT5LqzkQAS0FHHmCiJuo_0 2019-12-09T10:09:03.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.736+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.nfO3hAg-QUaXT5LqzkQAS0FHHmCiJuo_1 2019-12-09T10:09:03.778 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.784+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:03.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.900+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:09:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.951+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:03.961 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:03.968+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:03.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:09:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_Uev7uLtV9J03fd4YcqyPzib0NDCxpqa', time=2019-12-09T12:08:29.311270+0000, chain.objs.size()=1 2019-12-09T10:09:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.x40vw_6VJdAjt5MlPWEjBDbJReXnd2P_1 2019-12-09T10:09:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_lG2CWeqiMfNS73VW7jdCmY-VnzWVa2E', time=2019-12-09T12:08:29.835716+0000, chain.objs.size()=1 2019-12-09T10:09:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.TqxMC_YWeYjF7ekNhNjc7WTWQeZ8hRP_0 2019-12-09T10:09:03.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_J2cOMSBceoxIcY6jDqW5WBIeQDe2LhF', time=2019-12-09T12:08:32.708001+0000, chain.objs.size()=2 2019-12-09T10:09:03.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.W3_uKNOVbwaDkGZz_4LlmprdpwkaaiD_1 2019-12-09T10:09:03.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.000+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.W3_uKNOVbwaDkGZz_4LlmprdpwkaaiD_2 2019-12-09T10:09:04.103 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.110+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:04.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.216+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:09:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.272+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.301+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.321+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.321+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.109', time=2019-12-09T12:08:28.089403+0000, chain.objs.size()=2 2019-12-09T10:09:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.321+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.JRmphzF0gW9sTRLvbKFOyHfT4bf0-34_0 2019-12-09T10:09:04.315 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.321+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.JRmphzF0gW9sTRLvbKFOyHfT4bf0-34_1 2019-12-09T10:09:04.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.398+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.508+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:09:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.575+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:04.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.606+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.671+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.671+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.40', time=2019-12-09T12:08:23.082208+0000, chain.objs.size()=4 2019-12-09T10:09:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.671+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3__multipart_foo.2~2sX-xbxM7-wAL8KatxKKLMstYiN7vyA.1 2019-12-09T10:09:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.672+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3__multipart_foo.2~2sX-xbxM7-wAL8KatxKKLMstYiN7vyA.2 2019-12-09T10:09:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.672+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3__multipart_foo.2~2sX-xbxM7-wAL8KatxKKLMstYiN7vyA.3 2019-12-09T10:09:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.672+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.3__multipart_foo.2~2sX-xbxM7-wAL8KatxKKLMstYiN7vyA.4 2019-12-09T10:09:04.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.836+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:04.961 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:04.968+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:09:05.020 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.026+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:05.038 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.045+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:05.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.056+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:05.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.056+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_k1JMn1zJyDc-kOeoHxKX8qhNLpdqIn5', time=2019-12-09T12:08:28.268873+0000, chain.objs.size()=2 2019-12-09T10:09:05.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.056+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.4S1sJl3sZL6PcsrleOnmuJRao7gBO1b_0 2019-12-09T10:09:05.051 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.056+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.4S1sJl3sZL6PcsrleOnmuJRao7gBO1b_1 2019-12-09T10:09:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.155+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:05.256 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.263+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:09:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.319+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:05.338 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.344+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:05.354 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.361+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:05.400 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.406+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:09:05.458 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.465+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.483+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.203', time=2019-12-09T12:08:35.587308+0000, chain.objs.size()=5 2019-12-09T10:09:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-LUKEWARM.2~1tnmlaiKfC9ehf2Te1FNExuucpzRLge.1 2019-12-09T10:09:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__shadow_foo-LUKEWARM.2~1tnmlaiKfC9ehf2Te1FNExuucpzRLge.1_1 2019-12-09T10:09:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-LUKEWARM.2~1tnmlaiKfC9ehf2Te1FNExuucpzRLge.2 2019-12-09T10:09:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__shadow_foo-LUKEWARM.2~1tnmlaiKfC9ehf2Te1FNExuucpzRLge.2_1 2019-12-09T10:09:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.515+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-LUKEWARM.2~1tnmlaiKfC9ehf2Te1FNExuucpzRLge.3 2019-12-09T10:09:05.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.605+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:05.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.713+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:09:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.768+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.794+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.813+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.813+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.84', time=2019-12-09T12:08:37.172115+0000, chain.objs.size()=3 2019-12-09T10:09:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.813+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6__shadow_.39jWXb1DqcZ7WIcp3fWjsbor_nSmSQN_0 2019-12-09T10:09:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.813+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6__shadow_.39jWXb1DqcZ7WIcp3fWjsbor_nSmSQN_1 2019-12-09T10:09:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.813+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6__shadow_.39jWXb1DqcZ7WIcp3fWjsbor_nSmSQN_2 2019-12-09T10:09:05.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:05.900+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.041+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:09:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.108+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:06.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.138+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:06.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:09:06.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_8LIMi255n3XkXkIOeRYuloHVBCEERte', time=2019-12-09T12:08:29.647459+0000, chain.objs.size()=1 2019-12-09T10:09:06.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.AwuBzp5V_WlK1LnOXD3EBvXUDUrHCSr_0 2019-12-09T10:09:06.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.225', time=2019-12-09T12:08:32.299643+0000, chain.objs.size()=5 2019-12-09T10:09:06.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.1 2019-12-09T10:09:06.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.1_1 2019-12-09T10:09:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.2 2019-12-09T10:09:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.2_1 2019-12-09T10:09:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.166+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.3 2019-12-09T10:09:06.255 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.260+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:06.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.333+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:09:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.380+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:06.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.397+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:06.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:06.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.235', time=2019-12-09T12:08:33.275807+0000, chain.objs.size()=5 2019-12-09T10:09:06.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.1 2019-12-09T10:09:06.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.1_1 2019-12-09T10:09:06.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.2 2019-12-09T10:09:06.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.2_1 2019-12-09T10:09:06.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.409+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.3 2019-12-09T10:09:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.490+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.559+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:09:06.602 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.608+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:06.620 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.626+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.646+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:09:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.646+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.245', time=2019-12-09T12:08:34.499394+0000, chain.objs.size()=3 2019-12-09T10:09:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.646+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-STANDARD.2~G2V5yRXQ5xi5miuX_5anXe9M3TeSPEw.1 2019-12-09T10:09:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.646+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-STANDARD.2~G2V5yRXQ5xi5miuX_5anXe9M3TeSPEw.2 2019-12-09T10:09:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-STANDARD.2~G2V5yRXQ5xi5miuX_5anXe9M3TeSPEw.3 2019-12-09T10:09:06.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.196', time=2019-12-09T12:08:35.378509+0000, chain.objs.size()=5 2019-12-09T10:09:06.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-FROZEN.2~UCzKBRg1xvxQpa_FulN-aupIpQS3Lr1.1 2019-12-09T10:09:06.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__shadow_foo-FROZEN.2~UCzKBRg1xvxQpa_FulN-aupIpQS3Lr1.1_1 2019-12-09T10:09:06.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-FROZEN.2~UCzKBRg1xvxQpa_FulN-aupIpQS3Lr1.2 2019-12-09T10:09:06.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__shadow_foo-FROZEN.2~UCzKBRg1xvxQpa_FulN-aupIpQS3Lr1.2_1 2019-12-09T10:09:06.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.647+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-FROZEN.2~UCzKBRg1xvxQpa_FulN-aupIpQS3Lr1.3 2019-12-09T10:09:06.898 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:06.904+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:07.083 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.088+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:09:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.133+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.150+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:07.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.159+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.214+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:09:07.285 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.291+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.319 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.324+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:07.332 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.338+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.404 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.409+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:09:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.476+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.502 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.507+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:07.518 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.523+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.568 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.574+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:09:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.642+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.674+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:07.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:07.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.233', time=2019-12-09T12:08:32.918922+0000, chain.objs.size()=5 2019-12-09T10:09:07.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.1 2019-12-09T10:09:07.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.1_1 2019-12-09T10:09:07.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.2 2019-12-09T10:09:07.690 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.2_1 2019-12-09T10:09:07.690 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.693+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-LUKEWARM.2~hEDoH-lnvLowJ6HS39nZKdRDWoSd8ZV.3 2019-12-09T10:09:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.794+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:07.862 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.868+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:09:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.911+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:07.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.928+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:07.935 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.940+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:07.935 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.940+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.55', time=2019-12-09T12:08:24.478596+0000, chain.objs.size()=4 2019-12-09T10:09:07.935 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.940+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4__multipart_foo.2~Ayr1Eo_dGkvwQyHhV6vazKmgqZnKzeH.1 2019-12-09T10:09:07.936 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.941+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4__multipart_foo.2~Ayr1Eo_dGkvwQyHhV6vazKmgqZnKzeH.2 2019-12-09T10:09:07.936 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.941+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4__multipart_foo.2~Ayr1Eo_dGkvwQyHhV6vazKmgqZnKzeH.3 2019-12-09T10:09:07.936 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:07.941+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.4__multipart_foo.2~Ayr1Eo_dGkvwQyHhV6vazKmgqZnKzeH.4 2019-12-09T10:09:08.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.146+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:08.219 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.224+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:09:08.283 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.288+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.311+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:08.316 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.321+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:08.369 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.374+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:09:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.455+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:08.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.495+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.507+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:08.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.565+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:09:08.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.648+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.678+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.706+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:09:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.706+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.108', time=2019-12-09T12:08:27.851052+0000, chain.objs.size()=2 2019-12-09T10:09:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.706+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.JRmphzF0gW9sTRLvbKFOyHfT4bf0-34_0 2019-12-09T10:09:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.706+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.7__shadow_.JRmphzF0gW9sTRLvbKFOyHfT4bf0-34_1 2019-12-09T10:09:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_2OF0GHFmK9NWT-97S5xrE1Vyi8MXEAB', time=2019-12-09T12:08:32.466162+0000, chain.objs.size()=3 2019-12-09T10:09:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.7Wso5wOiUqfGMlfkXLg0GGtMDUGBEV6_0 2019-12-09T10:09:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.7Wso5wOiUqfGMlfkXLg0GGtMDUGBEV6_1 2019-12-09T10:09:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.7Wso5wOiUqfGMlfkXLg0GGtMDUGBEV6_2 2019-12-09T10:09:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.85', time=2019-12-09T12:08:37.373831+0000, chain.objs.size()=3 2019-12-09T10:09:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6__shadow_.IOW6xksMQSaf_zLHQX_Tbv6RxjBs8JT_0 2019-12-09T10:09:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6__shadow_.IOW6xksMQSaf_zLHQX_Tbv6RxjBs8JT_1 2019-12-09T10:09:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.707+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.6__shadow_.IOW6xksMQSaf_zLHQX_Tbv6RxjBs8JT_2 2019-12-09T10:09:08.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.826+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.909+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:09:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.968+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:08.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:08.996+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:09.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.009+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:09.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.009+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_PVyRD5UM-UP6YBldW0L1CO_v-GOeZ4l', time=2019-12-09T12:08:33.104406+0000, chain.objs.size()=3 2019-12-09T10:09:09.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.009+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.9yxnd31AtrwH6gMqhDZfWOAgW0R4Jh__0 2019-12-09T10:09:09.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.009+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.9yxnd31AtrwH6gMqhDZfWOAgW0R4Jh__1 2019-12-09T10:09:09.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.009+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.9yxnd31AtrwH6gMqhDZfWOAgW0R4Jh__2 2019-12-09T10:09:09.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.073+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:09.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.169+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:09:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.251+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:09.274 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.280+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.294+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:09.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.343+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:09:09.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.423+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:09.449 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.455+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.468+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:09.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.519+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:09:09.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.604+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.630+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:09.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.649+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:09.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.649+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.71', time=2019-12-09T12:08:26.032963+0000, chain.objs.size()=4 2019-12-09T10:09:09.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.649+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5__multipart_foo.2~jwq-G3xh8wF2c86cl7-b_PDKREINyfU.1 2019-12-09T10:09:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.649+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5__multipart_foo.2~jwq-G3xh8wF2c86cl7-b_PDKREINyfU.2 2019-12-09T10:09:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.649+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5__multipart_foo.2~jwq-G3xh8wF2c86cl7-b_PDKREINyfU.3 2019-12-09T10:09:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.649+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.5__multipart_foo.2~jwq-G3xh8wF2c86cl7-b_PDKREINyfU.4 2019-12-09T10:09:09.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.765+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:09.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.891+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:09:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:09.986+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:10.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.009+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.024+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:10.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.024+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_O8GsotdqQnKz9gygEiBuHdFhmsAgZuc', time=2019-12-09T12:08:33.552135+0000, chain.objs.size()=2 2019-12-09T10:09:10.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.024+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.l2aNxA0ZP6vIz9zjFcDgOC5H4dgXmxu_1 2019-12-09T10:09:10.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.024+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.l2aNxA0ZP6vIz9zjFcDgOC5H4dgXmxu_2 2019-12-09T10:09:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.179+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.280+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:09:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.328+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:10.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.351+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:10.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:10.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.224', time=2019-12-09T12:08:32.064367+0000, chain.objs.size()=5 2019-12-09T10:09:10.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.1 2019-12-09T10:09:10.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.1_1 2019-12-09T10:09:10.367 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.2 2019-12-09T10:09:10.367 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.2_1 2019-12-09T10:09:10.367 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.368+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__multipart_foo-FROZEN.2~FmRdxmYJ1A2Socv9HakcE7D3ymdCykv.3 2019-12-09T10:09:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.441+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:10.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.529+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:09:10.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.604+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:10.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.633+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.650+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.693+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:09:10.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.762+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:10.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.795+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:10.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.844+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:09:10.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.844+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_68KxTlJOIdEVN9J6oQflCha3W0ph-g5', time=2019-12-09T12:08:34.119219+0000, chain.objs.size()=1 2019-12-09T10:09:10.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.844+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.-n8jgpegGFB0rxBNi-jJw8BrZ_n1Cgu_0 2019-12-09T10:09:10.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:10.920+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.088+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:09:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.161+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:11.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.187+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.198+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:11.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.257+0000 7ffa5421bec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:09:11.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.334+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.371+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:09:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.20', time=2019-12-09T12:08:21.003687+0000, chain.objs.size()=4 2019-12-09T10:09:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2__multipart_foo.2~t9-hd-h2yQyHRk4amk2Y7flmEYFi_82.1 2019-12-09T10:09:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2__multipart_foo.2~t9-hd-h2yQyHRk4amk2Y7flmEYFi_82.2 2019-12-09T10:09:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2__multipart_foo.2~t9-hd-h2yQyHRk4amk2Y7flmEYFi_82.3 2019-12-09T10:09:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.2__multipart_foo.2~t9-hd-h2yQyHRk4amk2Y7flmEYFi_82.4 2019-12-09T10:09:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='_rH5av0TLo7BZ13AdlcnFhcXmAUSF6QT', time=2019-12-09T12:08:34.300396+0000, chain.objs.size()=1 2019-12-09T10:09:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.9__shadow_.EsFmajhjK7uwefMKESKWtyi0lpL0JIo_0 2019-12-09T10:09:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.210', time=2019-12-09T12:08:35.813140+0000, chain.objs.size()=3 2019-12-09T10:09:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-STANDARD.2~lHxinRnV4rSdBdHcKFFyDY2b2nN5wNJ.1 2019-12-09T10:09:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-STANDARD.2~lHxinRnV4rSdBdHcKFFyDY2b2nN5wNJ.2 2019-12-09T10:09:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.400+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.8__multipart_foo-STANDARD.2~lHxinRnV4rSdBdHcKFFyDY2b2nN5wNJ.3 2019-12-09T10:09:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.671+0000 7ffa5421bec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:09:11.802 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.808+0000 7ffa5421bec0 20 remove_watcher() i=0 2019-12-09T10:09:11.802 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.808+0000 7ffa5421bec0 2 removed watcher, disabling cache 2019-12-09T10:09:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.842+0000 7ffa5421bec0 20 remove_watcher() i=1 2019-12-09T10:09:11.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.885+0000 7ffa5421bec0 20 remove_watcher() i=2 2019-12-09T10:09:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.935+0000 7ffa5421bec0 20 remove_watcher() i=3 2019-12-09T10:09:11.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:11.963+0000 7ffa5421bec0 20 remove_watcher() i=4 2019-12-09T10:09:12.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.008+0000 7ffa5421bec0 20 remove_watcher() i=5 2019-12-09T10:09:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.037+0000 7ffa5421bec0 20 remove_watcher() i=6 2019-12-09T10:09:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.075+0000 7ffa5421bec0 20 remove_watcher() i=7 2019-12-09T10:09:12.079 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:09:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.203+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.234+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.234+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.239+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.239+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.243+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.243+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.255+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.421+0000 7fddd9172ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:09:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.421+0000 7fddd9172ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:09:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.421+0000 7fddd9172ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:09:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.421+0000 7fddd9172ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:09:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.421+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.432+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.432+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.440 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.445+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:09:12.440 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.445+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.460+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:12.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.460+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.479+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:09:12.474 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.479+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.488+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.488+0000 7fddd9172ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:09:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.488+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.500+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.500+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.516+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:12.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.516+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.529 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.534+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:09:12.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.534+0000 7fddd9172ec0 20 zonegroup default 2019-12-09T10:09:12.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.534+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.541+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.541+0000 7fddd9172ec0 10 Cannot find current period zone using local zone 2019-12-09T10:09:12.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.541+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.551+0000 7fddd9172ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:09:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.551+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.572+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:09:12.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.572+0000 7fddd9172ec0 20 rados->read ofs=0 len=0 2019-12-09T10:09:12.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.586+0000 7fddd9172ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:09:12.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.586+0000 7fddd9172ec0 20 zone default found 2019-12-09T10:09:12.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.586+0000 7fddd9172ec0 20 started sync module instance, tier type = 2019-12-09T10:09:12.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.586+0000 7fddd9172ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:09:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=0 2019-12-09T10:09:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=1 2019-12-09T10:09:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=2 2019-12-09T10:09:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=3 2019-12-09T10:09:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=4 2019-12-09T10:09:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=5 2019-12-09T10:09:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.968+0000 7fddd9172ec0 20 add_watcher() i=6 2019-12-09T10:09:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.998+0000 7fddd9172ec0 20 add_watcher() i=7 2019-12-09T10:09:12.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.998+0000 7fddd9172ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:09:12.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:12.998+0000 7fdd96fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:09:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:13.009+0000 7fddc6b9d700 0 -- 172.21.15.59:0/3364099627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564c4451e710 msgr2=0x564c4451daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:09:13.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:13.139+0000 7fddc639c700 0 -- 172.21.15.59:0/3415711147 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x564c4453eeb0 msgr2=0x564c445412d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:09:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.883+0000 7fdd057fa700 20 reqs_thread_entry: start 2019-12-09T10:09:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.883+0000 7fdcfeffd700 20 reqs_thread_entry: start 2019-12-09T10:09:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.883+0000 7fddd9172ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:09:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.884+0000 7fdcfd7fa700 20 reqs_thread_entry: start 2019-12-09T10:09:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.884+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:09:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.948+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.971+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:14.984+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.043+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:09:15.087 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.093+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.104 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.109+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:15.115 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.120+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.164+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:09:15.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.211+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.230+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:15.238 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.244+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.294+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:09:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.354+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.376+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:15.383 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.388+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.449 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.454+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:09:15.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.518+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.544+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.557+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.638+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:09:15.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.711+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.737 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.742+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:15.747 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.753+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.804+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:09:15.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.891+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:15.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.946+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:15.972+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.041 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.046+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:09:16.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.123+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.170+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.189+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.237+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:09:16.295 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.300+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.319 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.324+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.335+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.388+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:09:16.439 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.444+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.468+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:16.475 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.480+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.527+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:09:16.596 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.602+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.635+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:16.640 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.645+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.681 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.686+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:09:16.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.812+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.833+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:16.844 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.849+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:16.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.899+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:09:16.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:16.976+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.005+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.015 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.020+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.060 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.065+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:09:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.149+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.185 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.190+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.201+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.252+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:09:17.312 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.318+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.335 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.340+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.351+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.397+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:09:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.471+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.504+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.517 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.523+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.581+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:09:17.626 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.631+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.653+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.664+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.698 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.703+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:09:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.761+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.782+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.793+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.833+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:09:17.897 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.902+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.920 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.925+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.933+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:17.977 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:17.982+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:09:18.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.044+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.061 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.067+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.081+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.139+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:09:18.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.198+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.217+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:18.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.227+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.265 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.271+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:09:18.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.353+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.381+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.393+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.466+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:09:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.524+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.547 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.553+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.564+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.631+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:09:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.715+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.736+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.745+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.784 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.790+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:09:18.874 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.880+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.899+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:18.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.909+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:18.949+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:09:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.003+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.015 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.021+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.033+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.074+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:09:19.141 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.147+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.185+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.203+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.264+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:09:19.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.329+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.344 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.350+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.364+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.400 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.406+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:09:19.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.467+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.487+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.498+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.533 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.539+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:09:19.600 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.605+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.628+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.639+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.685+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:09:19.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.765+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.793+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.801 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.807+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.854 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.860+0000 7fddd9172ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:09:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.928+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.953+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:09:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:19.966+0000 7fddd9172ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:09:20.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.050+0000 7fddd9172ec0 20 remove_watcher() i=0 2019-12-09T10:09:20.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.050+0000 7fddd9172ec0 2 removed watcher, disabling cache 2019-12-09T10:09:20.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.092+0000 7fddd9172ec0 20 remove_watcher() i=1 2019-12-09T10:09:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.134+0000 7fddd9172ec0 20 remove_watcher() i=2 2019-12-09T10:09:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.193+0000 7fddd9172ec0 20 remove_watcher() i=3 2019-12-09T10:09:20.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.283+0000 7fddd9172ec0 20 remove_watcher() i=4 2019-12-09T10:09:20.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.343+0000 7fddd9172ec0 20 remove_watcher() i=5 2019-12-09T10:09:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.396+0000 7fddd9172ec0 20 remove_watcher() i=6 2019-12-09T10:09:20.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:09:20.467+0000 7fddd9172ec0 20 remove_watcher() i=7 2019-12-09T10:09:20.470 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-12-09T10:09:21.035 INFO:teuthology.orchestra.run.smithi059.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-12-09T10:09:24.133 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-12-09T10:09:24.333 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-12-09T10:09:25.435 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :clean UP-TO-DATE 2019-12-09T10:09:25.533 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-12-09T10:09:25.533 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-12-09T10:09:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-12-09T10:09:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-12-09T10:09:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-12-09T10:09:25.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-12-09T10:09:25.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-12-09T10:09:25.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-12-09T10:09:25.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-12-09T10:09:25.833 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-12-09T10:09:25.833 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-12-09T10:09:25.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-12-09T10:09:25.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-12-09T10:09:25.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-12-09T10:09:25.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-12-09T10:09:25.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-12-09T10:09:25.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-12-09T10:09:25.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-12-09T10:09:25.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-12-09T10:09:25.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-12-09T10:09:25.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-12-09T10:09:25.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-12-09T10:09:25.839 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-12-09T10:09:25.839 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-12-09T10:09:25.937 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-12-09T10:09:25.937 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-12-09T10:09:25.937 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-12-09T10:09:25.938 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-12-09T10:09:25.938 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-12-09T10:09:25.938 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-12-09T10:09:26.033 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-12-09T10:09:26.033 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-12-09T10:09:26.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-12-09T10:09:26.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-12-09T10:09:26.036 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-12-09T10:09:26.036 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-12-09T10:09:26.036 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-12-09T10:09:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-12-09T10:09:26.038 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-12-09T10:09:26.039 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-12-09T10:09:26.039 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-12-09T10:09:26.039 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-12-09T10:09:26.039 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-12-09T10:09:26.040 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-12-09T10:09:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-12-09T10:09:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-12-09T10:09:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-12-09T10:09:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-12-09T10:09:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-12-09T10:09:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-12-09T10:09:26.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-12-09T10:09:26.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-12-09T10:09:26.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-12-09T10:09:26.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-12-09T10:09:26.236 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-12-09T10:09:26.236 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-12-09T10:09:26.236 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-12-09T10:09:26.236 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-12-09T10:09:26.237 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-12-09T10:09:26.237 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-12-09T10:09:26.237 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-12-09T10:09:26.237 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-12-09T10:09:26.237 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-12-09T10:09:26.238 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-12-09T10:09:26.238 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-12-09T10:09:26.238 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-12-09T10:09:26.238 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-12-09T10:09:26.238 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-12-09T10:09:26.238 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-12-09T10:09:26.239 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-12-09T10:09:26.239 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-12-09T10:09:26.239 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-12-09T10:09:26.239 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-12-09T10:09:26.239 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-12-09T10:09:26.239 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-12-09T10:09:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-12-09T10:09:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-12-09T10:09:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-12-09T10:09:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-12-09T10:09:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-12-09T10:09:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-12-09T10:09:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-12-09T10:09:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-12-09T10:09:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-12-09T10:09:26.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-12-09T10:09:26.337 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-12-09T10:09:26.337 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-12-09T10:09:26.337 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-12-09T10:09:26.337 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-12-09T10:09:26.337 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-12-09T10:09:26.337 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-12-09T10:09:26.338 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-12-09T10:09:26.338 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-12-09T10:09:26.338 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-12-09T10:09:26.338 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-12-09T10:09:26.338 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-12-09T10:09:26.338 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-12-09T10:09:26.339 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-12-09T10:09:26.340 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-12-09T10:09:26.340 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-12-09T10:09:26.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-12-09T10:09:26.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-12-09T10:09:26.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-12-09T10:09:26.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-12-09T10:09:26.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-12-09T10:09:26.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-12-09T10:09:26.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-12-09T10:09:26.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-12-09T10:09:26.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-12-09T10:09:26.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-12-09T10:09:26.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-12-09T10:09:26.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-12-09T10:09:26.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-12-09T10:09:26.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-12-09T10:09:26.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-12-09T10:09:26.439 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-12-09T10:09:26.440 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-12-09T10:09:26.440 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-12-09T10:09:26.440 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-12-09T10:09:26.440 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-12-09T10:09:26.440 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-12-09T10:09:26.440 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-12-09T10:09:26.441 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-12-09T10:09:26.441 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-12-09T10:09:26.441 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-12-09T10:09:26.441 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-12-09T10:09:26.441 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-12-09T10:09:26.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-12-09T10:09:26.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-12-09T10:09:26.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-12-09T10:09:26.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-12-09T10:09:26.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-12-09T10:09:26.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-12-09T10:09:26.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-12-09T10:09:26.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-12-09T10:09:26.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-12-09T10:09:26.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-12-09T10:09:26.539 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-12-09T10:09:26.539 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-12-09T10:09:26.539 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-12-09T10:09:26.539 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-12-09T10:09:26.540 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-12-09T10:09:26.540 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-12-09T10:09:26.540 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-12-09T10:09:26.540 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-12-09T10:09:26.541 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-12-09T10:09:26.541 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-12-09T10:09:26.541 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-12-09T10:09:26.541 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-12-09T10:09:26.542 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-12-09T10:09:26.542 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-12-09T10:09:26.542 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-12-09T10:09:26.543 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-12-09T10:09:26.543 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-12-09T10:09:26.543 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-12-09T10:09:26.543 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-12-09T10:09:26.544 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-12-09T10:09:26.544 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-12-09T10:09:26.544 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-12-09T10:09:26.544 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-12-09T10:09:26.637 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-12-09T10:09:26.637 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-12-09T10:09:26.637 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-12-09T10:09:26.638 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-12-09T10:09:26.638 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-12-09T10:09:26.638 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-12-09T10:09:26.638 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-12-09T10:09:26.639 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-12-09T10:09:26.639 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-12-09T10:09:26.639 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-12-09T10:09:26.639 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-12-09T10:09:26.640 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-12-09T10:09:26.640 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-12-09T10:09:26.640 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-12-09T10:09:26.640 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-12-09T10:09:26.641 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-12-09T10:09:26.641 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-12-09T10:09:26.641 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-12-09T10:09:26.733 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-12-09T10:09:26.733 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-12-09T10:09:26.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-12-09T10:09:26.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-12-09T10:09:26.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-12-09T10:09:26.835 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-12-09T10:09:26.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-12-09T10:09:26.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-12-09T10:09:26.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-12-09T10:09:26.836 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-12-09T10:09:26.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-12-09T10:09:26.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-12-09T10:09:26.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-12-09T10:09:26.837 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-12-09T10:09:26.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-12-09T10:09:26.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-12-09T10:09:26.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-12-09T10:09:26.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-12-09T10:09:26.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-12-09T10:09:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-12-09T10:09:26.937 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-12-09T10:09:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-12-09T10:09:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-12-09T10:09:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-12-09T10:09:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-12-09T10:09:27.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-12-09T10:09:27.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-12-09T10:09:27.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-12-09T10:09:27.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-12-09T10:09:27.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-12-09T10:09:27.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-12-09T10:09:27.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-12-09T10:09:27.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-12-09T10:09:27.136 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-12-09T10:09:27.136 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-12-09T10:09:27.136 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-12-09T10:09:27.136 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-12-09T10:09:27.136 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-12-09T10:09:27.137 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-12-09T10:09:27.138 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-12-09T10:09:27.233 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-12-09T10:09:27.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-12-09T10:09:27.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-12-09T10:09:27.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-12-09T10:09:27.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-12-09T10:09:27.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-12-09T10:09:27.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-12-09T10:09:27.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-12-09T10:09:27.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-12-09T10:09:28.233 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-12-09T10:09:28.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-12-09T10:09:28.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-12-09T10:09:28.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-12-09T10:09:28.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-12-09T10:09:28.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-12-09T10:09:28.333 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-12-09T10:09:28.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-12-09T10:09:28.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-12-09T10:09:28.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-12-09T10:09:28.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-12-09T10:09:28.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-12-09T10:09:28.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-12-09T10:09:28.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-12-09T10:09:28.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-12-09T10:09:28.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-12-09T10:09:28.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-12-09T10:09:28.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-12-09T10:09:28.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-12-09T10:09:28.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-12-09T10:09:28.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-12-09T10:09:28.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-12-09T10:09:28.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-12-09T10:09:28.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-12-09T10:09:28.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-12-09T10:09:28.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-12-09T10:09:28.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-12-09T10:09:28.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-12-09T10:09:28.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-12-09T10:09:28.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-12-09T10:09:28.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-12-09T10:09:28.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-12-09T10:09:28.534 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-12-09T10:09:28.534 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-12-09T10:09:28.534 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-12-09T10:09:28.534 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-12-09T10:09:28.535 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-12-09T10:09:28.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-12-09T10:09:28.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-12-09T10:09:28.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-12-09T10:09:28.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-12-09T10:09:28.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-12-09T10:09:28.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-12-09T10:09:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-12-09T10:09:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-12-09T10:09:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-12-09T10:09:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-12-09T10:09:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-12-09T10:09:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-12-09T10:09:28.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-12-09T10:09:28.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-12-09T10:09:28.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-12-09T10:09:28.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-12-09T10:09:28.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-12-09T10:09:28.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-12-09T10:09:28.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-12-09T10:09:28.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-12-09T10:09:28.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-12-09T10:09:28.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-12-09T10:09:28.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-12-09T10:09:28.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-12-09T10:09:28.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-12-09T10:09:28.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-12-09T10:09:28.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-12-09T10:09:28.733 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-12-09T10:09:28.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-12-09T10:09:28.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-12-09T10:09:28.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-12-09T10:09:28.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-12-09T10:09:28.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-12-09T10:09:28.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-12-09T10:09:28.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-12-09T10:09:28.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-12-09T10:09:28.735 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-12-09T10:09:28.833 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-12-09T10:09:28.833 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-12-09T10:09:28.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-12-09T10:09:28.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-12-09T10:09:28.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-12-09T10:09:28.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-12-09T10:09:28.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-12-09T10:09:28.834 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-12-09T10:09:28.835 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-12-09T10:09:28.835 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-12-09T10:09:28.835 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-12-09T10:09:28.933 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-12-09T10:09:28.934 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-12-09T10:09:28.934 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-12-09T10:09:28.934 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-12-09T10:09:28.934 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-12-09T10:09:28.934 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-12-09T10:09:28.934 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-12-09T10:09:28.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-12-09T10:09:28.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-12-09T10:09:28.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-12-09T10:09:28.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-12-09T10:09:28.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-12-09T10:09:28.935 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-12-09T10:09:28.936 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-12-09T10:09:29.033 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-12-09T10:09:29.033 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-12-09T10:09:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-12-09T10:09:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-12-09T10:09:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-12-09T10:09:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-12-09T10:09:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-12-09T10:09:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-12-09T10:09:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-12-09T10:09:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-12-09T10:09:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-12-09T10:09:29.133 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-12-09T10:09:29.133 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-12-09T10:09:29.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-12-09T10:09:29.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-12-09T10:09:29.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-12-09T10:09:29.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-12-09T10:09:29.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-12-09T10:09:29.134 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-12-09T10:09:29.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-12-09T10:09:29.135 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-12-09T10:09:29.233 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-12-09T10:09:29.233 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-12-09T10:09:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-12-09T10:09:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-12-09T10:09:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-12-09T10:09:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-12-09T10:09:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-12-09T10:09:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-12-09T10:09:29.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-12-09T10:09:29.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-12-09T10:09:29.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-12-09T10:09:29.235 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-12-09T10:09:29.333 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-12-09T10:09:29.334 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-12-09T10:09:29.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-12-09T10:09:29.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-12-09T10:09:29.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-12-09T10:09:29.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-12-09T10:09:29.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-12-09T10:09:29.335 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-12-09T10:09:29.336 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-12-09T10:09:29.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-12-09T10:09:29.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-12-09T10:09:29.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-12-09T10:09:29.434 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-12-09T10:09:29.435 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-12-09T10:09:29.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-12-09T10:09:29.436 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-12-09T10:09:29.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-12-09T10:09:29.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-12-09T10:09:29.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-12-09T10:09:29.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-12-09T10:09:29.437 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-12-09T10:09:29.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-12-09T10:09:29.438 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-12-09T10:09:29.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-12-09T10:09:29.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-12-09T10:09:29.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-12-09T10:09:29.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-12-09T10:09:29.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-12-09T10:09:29.536 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-12-09T10:09:29.537 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-12-09T10:09:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-12-09T10:09:29.539 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-12-09T10:09:29.633 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-12-09T10:09:29.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-12-09T10:09:29.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-12-09T10:09:29.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-12-09T10:09:29.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-12-09T10:09:29.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-12-09T10:09:29.634 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-12-09T10:09:29.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-12-09T10:09:29.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-12-09T10:09:29.635 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-12-09T10:09:29.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-12-09T10:09:29.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-12-09T10:09:29.636 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-12-09T10:09:29.733 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-12-09T10:09:29.733 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-12-09T10:09:29.733 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-12-09T10:09:29.734 INFO:teuthology.orchestra.run.smithi059.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-12-09T10:09:38.734 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :compileJava 2019-12-09T10:09:38.735 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :processResources 2019-12-09T10:09:38.735 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :classes 2019-12-09T10:09:39.234 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:09:39.234 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :compileTestJava 2019-12-09T10:09:39.234 INFO:teuthology.orchestra.run.smithi059.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T10:09:39.235 INFO:teuthology.orchestra.run.smithi059.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T10:09:39.235 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:09:39.235 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T10:09:39.235 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :testClasses 2019-12-09T10:09:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:09:47.234 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :test 2019-12-09T10:09:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:09:47.234 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-12-09T10:10:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:10:47.837 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-12-09T10:10:54.435 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:10:54.436 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-12-09T10:10:57.736 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:10:57.738 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-12-09T10:11:05.036 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:11:05.036 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-12-09T10:12:20.338 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:12:20.339 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-12-09T10:12:27.938 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:12:27.938 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T10:12:28.438 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:12:28.438 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T10:12:29.438 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:12:29.438 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-12-09T10:12:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:12:32.238 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-12-09T10:12:40.141 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:12:40.141 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-12-09T10:13:26.739 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:13:26.739 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-12-09T10:13:41.739 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:13:41.739 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-12-09T10:15:08.633 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:32:05.254 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.642 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:32:05.263 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.648 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:32:05.269 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.655 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:32:05.276 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.661 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:32:05.262 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.676 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:32:05.277 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.918 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:32:05.538 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.926 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:32:05.547 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:15:08.961 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:32:05.561 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:18:30.943 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:18:30.943 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-12-09T10:18:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:18:47.943 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-12-09T10:18:50.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:18:50.443 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-12-09T10:18:58.543 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:18:58.543 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-12-09T10:19:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:19:03.743 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-12-09T10:19:17.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:19:17.043 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-12-09T10:20:15.444 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:20:15.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-12-09T10:20:20.823 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:37:17.440 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:20.836 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:37:17.453 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:20.840 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:37:17.457 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:20.842 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:37:17.441 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:20.844 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:37:17.461 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:20.847 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:37:17.446 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:21.017 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:37:17.632 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:21.018 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:37:17.613 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:21.101 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:37:17.718 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:22.844 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:20:22.844 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T10:20:23.244 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:20:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T10:20:24.144 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:20:24.144 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-12-09T10:21:08.579 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T10:21:08.522+0000 9af2d700 -1 handle_data failed with exit code -104 2019-12-09T10:21:09.844 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:21:09.844 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-12-09T10:21:19.544 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:21:19.544 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T10:22:21.344 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:22:21.344 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T10:22:36.349 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:22:36.349 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-12-09T10:23:08.944 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:23:08.944 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-12-09T10:23:17.943 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:23:17.944 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-12-09T10:25:32.407 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:42:29.025 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.424 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:42:29.042 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.434 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:42:29.052 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.438 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:42:29.038 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.446 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:42:29.045 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.451 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:42:29.069 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.616 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:42:29.234 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.681 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:42:29.299 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:32.714 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:42:29.313 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.422 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:42:39.040 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.429 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:42:39.047 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.455 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:42:39.073 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.456 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:42:39.073 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.480 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:42:39.098 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.500 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:42:39.118 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.518 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:42:39.117 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.520 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:42:39.120 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:25:42.549 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:42:39.148 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.147 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:47:39.766 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.152 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:47:39.771 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.167 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:47:39.786 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.172 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:47:39.791 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.175 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:47:39.774 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.182 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:47:39.781 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.357 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:47:39.976 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.361 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:47:39.980 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:30:43.432 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:47:40.031 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.343 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:51:00.960 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.364 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:51:00.982 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.377 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:51:00.996 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.392 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:51:01.010 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.395 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:51:00.993 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.424 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:51:01.022 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.470 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:51:01.089 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.480 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:51:01.098 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:34:04.587 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:51:01.185 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.736 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:52:51.354 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.740 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:52:51.359 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.749 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:52:51.368 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.750 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:52:51.348 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.752 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:52:51.371 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.753 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:52:51.351 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.848 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:52:51.446 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.857 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:52:51.475 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:35:54.868 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:52:51.487 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:01.343 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:37:01.343 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-12-09T10:37:11.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:37:11.443 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-12-09T10:38:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:38:47.445 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-12-09T10:38:58.343 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:38:58.343 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-12-09T10:40:06.944 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:40:06.945 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-12-09T10:40:16.743 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:40:16.744 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-12-09T10:40:34.344 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:40:34.344 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-12-09T10:40:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:40:55.344 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-12-09T10:40:58.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:40:58.044 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-12-09T10:41:05.713 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:58:02.331 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:05.721 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:58:02.339 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:05.735 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:58:02.352 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:05.737 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:58:02.354 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:05.740 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:58:02.338 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:05.769 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:58:02.367 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:06.002 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:58:02.619 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:06.028 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:58:02.645 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:06.086 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:58:02.684 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:41:17.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:41:17.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-12-09T10:41:25.444 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:41:25.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-12-09T10:41:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:41:47.344 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-12-09T10:41:50.844 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:41:50.844 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-12-09T10:42:18.444 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:42:18.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-12-09T10:42:25.883 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:59:22.500 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:25.935 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:59:22.552 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:25.971 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:59:22.588 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:25.973 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:59:22.590 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:25.975 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:59:22.573 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:25.980 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:59:22.578 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.001 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:00:59:22.618 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.026 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:00:59:22.643 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.071 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:00:59:22.668 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:30.944 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:42:30.944 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-12-09T10:43:22.444 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:43:22.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-12-09T10:43:34.644 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:43:34.644 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-12-09T10:45:07.144 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:45:07.144 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-12-09T10:45:23.344 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:45:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-12-09T10:46:17.834 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:03:14.451 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.838 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:03:14.455 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.846 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:03:14.443 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.847 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:03:14.446 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.849 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:03:14.466 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.855 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:03:14.472 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.979 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:03:14.577 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.997 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:03:14.614 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:17.999 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:03:14.616 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:46:35.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:46:35.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-12-09T10:46:58.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:46:58.044 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-12-09T10:48:28.843 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:48:28.844 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-12-09T10:50:47.481 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:07:44.098 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.515 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:07:44.132 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.521 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:07:44.137 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.521 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:07:44.139 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.552 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:07:44.150 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.559 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:07:44.158 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.707 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:07:44.324 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.718 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:07:44.317 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:50:47.771 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:07:44.388 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.720 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:08:26.337 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.727 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:08:26.344 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.735 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:08:26.333 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.740 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:08:26.357 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.741 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:08:26.340 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.744 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:08:26.362 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.788 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:08:26.405 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.790 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:08:26.408 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:51:29.809 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:08:26.408 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:41.892 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:13:38.509 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:41.896 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:13:38.513 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:41.904 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:13:38.521 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:41.907 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:13:38.524 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:41.915 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:13:38.513 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:41.921 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:13:38.520 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:42.090 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:13:38.707 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:42.101 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:13:38.700 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:42.114 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:13:38.731 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:56:58.443 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:56:58.444 INFO:teuthology.orchestra.run.smithi059.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-12-09T10:57:39.243 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:57:39.244 INFO:teuthology.orchestra.run.smithi059.stdout:---------------------------------------------------------------------- 2019-12-09T10:57:39.244 INFO:teuthology.orchestra.run.smithi059.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-12-09T10:57:39.244 INFO:teuthology.orchestra.run.smithi059.stdout:---------------------------------------------------------------------- 2019-12-09T10:57:39.407 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:57:39.408 INFO:teuthology.orchestra.run.smithi059.stdout:BUILD SUCCESSFUL in 48m 18s 2019-12-09T10:57:39.408 INFO:teuthology.orchestra.run.smithi059.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-12-09T10:57:39.727 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:57:39.880 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.875+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.905+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.905+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.916+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.916+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:39.925 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.920+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:39.925 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.920+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:39.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:39.945+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:40.188 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.182+0000 7f45f35c9ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:57:40.188 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.182+0000 7f45f35c9ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:57:40.188 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.182+0000 7f45f35c9ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:57:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.182+0000 7f45f35c9ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:57:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.182+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.202+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.202+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.220 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.215+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:57:40.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.215+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.227+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.227+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.240+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:57:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.240+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.247+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.247+0000 7f45f35c9ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:57:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.247+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.257+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.257+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.272+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:40.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.272+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.291+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:57:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.291+0000 7f45f35c9ec0 20 zonegroup default 2019-12-09T10:57:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.291+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.300+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.300+0000 7f45f35c9ec0 10 Cannot find current period zone using local zone 2019-12-09T10:57:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.300+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.311+0000 7f45f35c9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.311+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.331+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:40.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.331+0000 7f45f35c9ec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.351+0000 7f45f35c9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:57:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.351+0000 7f45f35c9ec0 20 zone default found 2019-12-09T10:57:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.351+0000 7f45f35c9ec0 20 started sync module instance, tier type = 2019-12-09T10:57:40.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.351+0000 7f45f35c9ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:57:40.816 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=0 2019-12-09T10:57:40.816 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=1 2019-12-09T10:57:40.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=2 2019-12-09T10:57:40.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=3 2019-12-09T10:57:40.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=4 2019-12-09T10:57:40.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=5 2019-12-09T10:57:40.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.810+0000 7f45f35c9ec0 20 add_watcher() i=6 2019-12-09T10:57:40.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.837+0000 7f45f35c9ec0 20 add_watcher() i=7 2019-12-09T10:57:40.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.837+0000 7f45f35c9ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:57:40.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.837+0000 7f45b0fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:57:40.849 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:40.844+0000 7f45dfff2700 0 -- 172.21.15.59:0/2489878325 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557e172435b0 msgr2=0x557e1724a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T10:57:41.051 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:41.046+0000 7f45dfff2700 0 --2- 172.21.15.59:0/2489878325 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557e172435b0 0x557e1724a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:57:41.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:41.087+0000 7f45e0ff4700 0 -- 172.21.15.59:0/927716023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557e17224d70 msgr2=0x557e1721aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:57:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.454+0000 7f4522ffd700 20 reqs_thread_entry: start 2019-12-09T10:57:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.454+0000 7f4520ff9700 20 reqs_thread_entry: start 2019-12-09T10:57:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.454+0000 7f45f35c9ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:57:42.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.455+0000 7f4516ffd700 20 reqs_thread_entry: start 2019-12-09T10:57:42.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.455+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:57:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.506+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.526+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.537+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.569+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:57:42.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.620+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.638+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:42.652 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.647+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.686+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:57:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.746+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.774 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.768+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:42.782 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.777+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.844 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.838+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:57:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.901+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.929+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.945+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:42.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:42.987+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:57:43.076 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.070+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.101+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.438', time=2019-12-09T12:16:08.075220+0000, chain.objs.size()=9 2019-12-09T10:57:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.1 2019-12-09T10:57:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.1_1 2019-12-09T10:57:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.2 2019-12-09T10:57:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.2_1 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.3 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.3_1 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.4 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.4_1 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~7uyN2db5d-Ieel_KEO1hqvMCJxSAINB.5 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.652', time=2019-12-09T12:38:50.287454+0000, chain.objs.size()=9 2019-12-09T10:57:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__multipart_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.1 2019-12-09T10:57:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.125+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__shadow_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.1_1 2019-12-09T10:57:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.126+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__multipart_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.2 2019-12-09T10:57:43.385 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.379+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__shadow_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.2_1 2019-12-09T10:57:43.385 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.379+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__multipart_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.3 2019-12-09T10:57:43.385 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.379+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__shadow_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.3_1 2019-12-09T10:57:43.519 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.514+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__multipart_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.4 2019-12-09T10:57:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.514+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__shadow_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.4_1 2019-12-09T10:57:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.514+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.29__multipart_key1.2~ut2ggnGMmfpjJFVxTRXqZyMJoY5AOlX.5 2019-12-09T10:57:43.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.762+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.851+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:57:43.922 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.916+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.938+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.949+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:43.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:43.984+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:57:44.046 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.041+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.069+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.509', time=2019-12-09T12:20:18.125871+0000, chain.objs.size()=9 2019-12-09T10:57:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__multipart_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.1 2019-12-09T10:57:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__shadow_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.1_1 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__multipart_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.2 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__shadow_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.2_1 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__multipart_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.3 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__shadow_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.3_1 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__multipart_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.4 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__shadow_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.4_1 2019-12-09T10:57:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.106+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.24__multipart_key1.2~MjryCdi_WIj87o-EGR-F6-iVn5R5OLn.5 2019-12-09T10:57:44.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.521+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.667+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:57:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.710+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.727+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:44.755 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.629', time=2019-12-09T12:36:59.853109+0000, chain.objs.size()=16 2019-12-09T10:57:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.1 2019-12-09T10:57:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.1_1 2019-12-09T10:57:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.1_2 2019-12-09T10:57:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.2 2019-12-09T10:57:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.2_1 2019-12-09T10:57:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.2_2 2019-12-09T10:57:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.3 2019-12-09T10:57:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.3_1 2019-12-09T10:57:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.3_2 2019-12-09T10:57:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.4 2019-12-09T10:57:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.4_1 2019-12-09T10:57:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:44.749+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.4_2 2019-12-09T10:57:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.006+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.5 2019-12-09T10:57:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.168+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.5_1 2019-12-09T10:57:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.168+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.5_2 2019-12-09T10:57:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.169+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~H1ArLYSFSJZ3fCoDWuFDSyNxFWaE2Ee.6 2019-12-09T10:57:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.392+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:45.524 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.518+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:57:45.572 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.566+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:45.590 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.584+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:45.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.593+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:45.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.627+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:57:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.692+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.710+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:45.741 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:45.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ', time=2019-12-09T12:40:50.612004+0000, chain.objs.size()=32 2019-12-09T10:57:45.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.1 2019-12-09T10:57:45.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.2 2019-12-09T10:57:45.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.3 2019-12-09T10:57:45.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.4 2019-12-09T10:57:45.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.5 2019-12-09T10:57:45.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.6 2019-12-09T10:57:45.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.7 2019-12-09T10:57:45.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.8 2019-12-09T10:57:45.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.9 2019-12-09T10:57:45.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.10 2019-12-09T10:57:45.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.11 2019-12-09T10:57:45.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.735+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.12 2019-12-09T10:57:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.924+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.13 2019-12-09T10:57:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.14 2019-12-09T10:57:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.dGnGLxDUL2FMh9pTL-dziZPBpYbi1B5.15 2019-12-09T10:57:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1._8VCXpg9KC-9avjG2IizGH1LrVufk9N.16 2019-12-09T10:57:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.QNgkEibDzS2B_HTxbPWfKECys88PcpC.17 2019-12-09T10:57:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.jCoHQql_p7V3QlNzN7sOxwkv6oL1UyT.18 2019-12-09T10:57:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.lPUhuF3PZIGH2jd8xnw5pam2_BEhtOj.19 2019-12-09T10:57:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.20 2019-12-09T10:57:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.QvHWTRJh-ME9C1h8-omlxyYJN57-YxW.21 2019-12-09T10:57:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.22 2019-12-09T10:57:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:45.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.23 2019-12-09T10:57:46.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.118+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.24 2019-12-09T10:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.241+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.25 2019-12-09T10:57:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.254+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.26 2019-12-09T10:57:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.254+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.27 2019-12-09T10:57:46.344 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.28 2019-12-09T10:57:46.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.339+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.29 2019-12-09T10:57:46.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.339+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.30 2019-12-09T10:57:46.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.339+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.31 2019-12-09T10:57:46.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.339+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.31__multipart_key1.2~ZayEZt7EPzHRnTMfyRlT4jes4a3hLOZ.33 2019-12-09T10:57:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.591+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:46.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.663+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:57:46.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.701+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.717+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.446', time=2019-12-09T12:17:07.744529+0000, chain.objs.size()=9 2019-12-09T10:57:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.1 2019-12-09T10:57:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.1_1 2019-12-09T10:57:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.2 2019-12-09T10:57:46.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.2_1 2019-12-09T10:57:46.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.3 2019-12-09T10:57:46.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.3_1 2019-12-09T10:57:46.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.4 2019-12-09T10:57:46.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.4_1 2019-12-09T10:57:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~FQpYrewYi3WcKmST52IMvJ-weuwBACN.5 2019-12-09T10:57:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.601', time=2019-12-09T12:30:39.124279+0000, chain.objs.size()=21 2019-12-09T10:57:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.1 2019-12-09T10:57:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.737+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.1_1 2019-12-09T10:57:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.738+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.2 2019-12-09T10:57:46.983 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.977+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.2_1 2019-12-09T10:57:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.981+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.3 2019-12-09T10:57:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.981+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.3_1 2019-12-09T10:57:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.981+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.4 2019-12-09T10:57:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.981+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.4_1 2019-12-09T10:57:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.981+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.5 2019-12-09T10:57:46.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:46.981+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.5_1 2019-12-09T10:57:47.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.181+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.6 2019-12-09T10:57:47.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.181+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.6_1 2019-12-09T10:57:47.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.181+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.7 2019-12-09T10:57:47.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.181+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.7_1 2019-12-09T10:57:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.8 2019-12-09T10:57:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.8_1 2019-12-09T10:57:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.9 2019-12-09T10:57:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.9_1 2019-12-09T10:57:47.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.10 2019-12-09T10:57:47.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.10_1 2019-12-09T10:57:47.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.353+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~FgJQK5gVwqCZDS2A7Pd-1eRo07y7TkR.11 2019-12-09T10:57:47.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.604+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.747+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:57:47.802 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.796+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:47.827 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.821+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:47.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:47.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.454', time=2019-12-09T12:17:31.385914+0000, chain.objs.size()=9 2019-12-09T10:57:47.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.1 2019-12-09T10:57:47.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.1_1 2019-12-09T10:57:47.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.2 2019-12-09T10:57:47.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.2_1 2019-12-09T10:57:47.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.3 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.3_1 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.4 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.4_1 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~xunq8JyQ6mtDeRGBfKqwQU1zbMfSJ_X.5 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.613', time=2019-12-09T12:33:07.294478+0000, chain.objs.size()=19 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.1 2019-12-09T10:57:47.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.1_1 2019-12-09T10:57:47.859 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:47.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.2 2019-12-09T10:57:48.082 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.076+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.2_1 2019-12-09T10:57:48.122 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.3 2019-12-09T10:57:48.122 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.3_1 2019-12-09T10:57:48.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.4 2019-12-09T10:57:48.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.4_1 2019-12-09T10:57:48.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.5 2019-12-09T10:57:48.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.5_1 2019-12-09T10:57:48.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.6 2019-12-09T10:57:48.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.6_1 2019-12-09T10:57:48.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.116+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.7 2019-12-09T10:57:48.156 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.150+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.7_1 2019-12-09T10:57:48.344 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.339+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.8 2019-12-09T10:57:48.414 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.408+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.8_1 2019-12-09T10:57:48.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.411+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.9 2019-12-09T10:57:48.460 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.454+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.9_1 2019-12-09T10:57:48.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.470+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~ZzBy0wrGK9lUVR8y8U7egxFLXpICDdI.10 2019-12-09T10:57:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.623+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.703+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:57:48.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.738+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:48.758 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.752+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.466', time=2019-12-09T12:18:29.174170+0000, chain.objs.size()=7 2019-12-09T10:57:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.1 2019-12-09T10:57:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.1_1 2019-12-09T10:57:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.1_2 2019-12-09T10:57:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.2 2019-12-09T10:57:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.2_1 2019-12-09T10:57:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.2_2 2019-12-09T10:57:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~Gquleed8LC9yQBc0bT1bOSYn2WB1YaS.3 2019-12-09T10:57:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.766+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~1BEngcKrmsraDZA8xTDdAdoZpSSm2gd', time=2019-12-09T12:57:31.369062+0000, chain.objs.size()=0 2019-12-09T10:57:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:48.964+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:49.087 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.081+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:57:49.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.128+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:49.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.148+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:49.180 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:49.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.588', time=2019-12-09T12:28:29.772686+0000, chain.objs.size()=21 2019-12-09T10:57:49.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.1 2019-12-09T10:57:49.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.1_1 2019-12-09T10:57:49.181 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.2 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.2_1 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.3 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.3_1 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.4 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.4_1 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.5 2019-12-09T10:57:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.5_1 2019-12-09T10:57:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.6 2019-12-09T10:57:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.174+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.6_1 2019-12-09T10:57:49.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.403+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.7 2019-12-09T10:57:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.7_1 2019-12-09T10:57:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.8 2019-12-09T10:57:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.8_1 2019-12-09T10:57:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.9 2019-12-09T10:57:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.9_1 2019-12-09T10:57:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.10 2019-12-09T10:57:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.432+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.10_1 2019-12-09T10:57:49.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.475+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~o7wVZ_NK5Y0St-U_lqtt8BOVcKAKNrR.11 2019-12-09T10:57:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.475+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.637', time=2019-12-09T12:37:04.847539+0000, chain.objs.size()=16 2019-12-09T10:57:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.475+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.1 2019-12-09T10:57:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.475+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.1_1 2019-12-09T10:57:49.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.689+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.1_2 2019-12-09T10:57:49.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.689+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.2 2019-12-09T10:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.701+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.2_1 2019-12-09T10:57:49.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.716+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.2_2 2019-12-09T10:57:49.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.716+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.3 2019-12-09T10:57:49.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.750+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.3_1 2019-12-09T10:57:49.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.750+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.3_2 2019-12-09T10:57:49.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.750+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.4 2019-12-09T10:57:49.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.890+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.4_1 2019-12-09T10:57:49.896 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.890+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.4_2 2019-12-09T10:57:49.896 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.890+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.5 2019-12-09T10:57:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.935+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.5_1 2019-12-09T10:57:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.935+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.5_2 2019-12-09T10:57:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:49.935+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~53h5vTvfT878ny99fgAunn7gZF42KVU.6 2019-12-09T10:57:50.274 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.268+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.378+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:57:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.430+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.456+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:50.478 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq', time=2019-12-09T12:10:47.100814+0000, chain.objs.size()=9 2019-12-09T10:57:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__multipart_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.1 2019-12-09T10:57:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__shadow_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.1_1 2019-12-09T10:57:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__multipart_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.2 2019-12-09T10:57:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__shadow_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.2_1 2019-12-09T10:57:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__multipart_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.3 2019-12-09T10:57:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__shadow_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.3_1 2019-12-09T10:57:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__multipart_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.4 2019-12-09T10:57:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__shadow_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.4_1 2019-12-09T10:57:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.13__multipart_key1.2~YAb0JYqbITkNsJKXMVEaMaq1oro5bcq.5 2019-12-09T10:57:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.430', time=2019-12-09T12:18:41.799248+0000, chain.objs.size()=5 2019-12-09T10:57:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.19__shadow_.NPlaDdjrOWmTMpmG6OxIXGa3gMrwfwk_1 2019-12-09T10:57:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.19__shadow_.NPlaDdjrOWmTMpmG6OxIXGa3gMrwfwk_2 2019-12-09T10:57:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.472+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.19__shadow_.NPlaDdjrOWmTMpmG6OxIXGa3gMrwfwk_3 2019-12-09T10:57:50.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.647+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.19__shadow_.NPlaDdjrOWmTMpmG6OxIXGa3gMrwfwk_4 2019-12-09T10:57:50.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.689+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.19__shadow_.NPlaDdjrOWmTMpmG6OxIXGa3gMrwfwk_5 2019-12-09T10:57:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.849+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.954+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:57:50.998 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:50.992+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:51.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.013+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.023+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.061+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:57:51.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.131+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:51.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.155+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:51.173 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.168+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:51.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.208+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:57:51.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.288+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.312+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:51.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:51.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~qB8z9JCxZxL0CCVk6mKADzphDt4Hn-C', time=2019-12-09T12:23:13.297815+0000, chain.objs.size()=5 2019-12-09T10:57:51.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.27__multipart_key1.2~qB8z9JCxZxL0CCVk6mKADzphDt4Hn-C.7 2019-12-09T10:57:51.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.27__shadow_key1.2~qB8z9JCxZxL0CCVk6mKADzphDt4Hn-C.7_1 2019-12-09T10:57:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.27__multipart_key1.2~qB8z9JCxZxL0CCVk6mKADzphDt4Hn-C.10 2019-12-09T10:57:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.27__shadow_key1.2~qB8z9JCxZxL0CCVk6mKADzphDt4Hn-C.10_1 2019-12-09T10:57:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.27__multipart_key1.2~qB8z9JCxZxL0CCVk6mKADzphDt4Hn-C.13 2019-12-09T10:57:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.621', time=2019-12-09T12:35:18.378766+0000, chain.objs.size()=16 2019-12-09T10:57:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.1 2019-12-09T10:57:51.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.1_1 2019-12-09T10:57:51.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.1_2 2019-12-09T10:57:51.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.2 2019-12-09T10:57:51.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.2_1 2019-12-09T10:57:51.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.2_2 2019-12-09T10:57:51.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.338+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.3 2019-12-09T10:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.584+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.3_1 2019-12-09T10:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.584+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.3_2 2019-12-09T10:57:51.732 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.4 2019-12-09T10:57:51.732 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.4_1 2019-12-09T10:57:51.732 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.4_2 2019-12-09T10:57:51.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.5 2019-12-09T10:57:51.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.5_1 2019-12-09T10:57:51.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__shadow_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.5_2 2019-12-09T10:57:51.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.726+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.28__multipart_key1.2~2tR7lmIaTc7DzaiJqJKbI1eUgYG15XC.6 2019-12-09T10:57:51.972 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:51.964+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.043+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:57:52.097 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.091+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:52.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.109+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:57:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.417', time=2019-12-09T12:13:28.842885+0000, chain.objs.size()=9 2019-12-09T10:57:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.1 2019-12-09T10:57:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.1_1 2019-12-09T10:57:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.2 2019-12-09T10:57:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.2_1 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.3 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.3_1 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.4 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.4_1 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.5 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.415', time=2019-12-09T12:13:35.180773+0000, chain.objs.size()=9 2019-12-09T10:57:52.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.1 2019-12-09T10:57:52.136 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.1_1 2019-12-09T10:57:52.136 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.127+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.2 2019-12-09T10:57:52.216 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.210+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.2_1 2019-12-09T10:57:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.391+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.3 2019-12-09T10:57:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.391+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.3_1 2019-12-09T10:57:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.391+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.4 2019-12-09T10:57:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.391+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__shadow_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.4_1 2019-12-09T10:57:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.391+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.17__multipart_key1.2~JQETyoHooFsosFwKQyiQE5WuhZSIG9s.5 2019-12-09T10:57:52.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.677+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.742+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:57:52.786 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.780+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:52.801 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.795+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:52.811 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.804+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:52.844 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.838+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:57:52.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.873+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:52.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.888+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.898+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.898+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.545', time=2019-12-09T12:22:26.983819+0000, chain.objs.size()=2 2019-12-09T10:57:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.898+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__shadow_.mx_kxP_AjUFUCE3FhPmRKat5h2gRjsl_1 2019-12-09T10:57:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.898+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__shadow_.mx_kxP_AjUFUCE3FhPmRKat5h2gRjsl_2 2019-12-09T10:57:53.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:52.995+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:53.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.080+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:57:53.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.118+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:53.139 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.133+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:53.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.667', time=2019-12-09T12:40:09.692470+0000, chain.objs.size()=9 2019-12-09T10:57:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__multipart_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.1 2019-12-09T10:57:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__shadow_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.1_1 2019-12-09T10:57:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__multipart_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.2 2019-12-09T10:57:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__shadow_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.2_1 2019-12-09T10:57:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__multipart_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.3 2019-12-09T10:57:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__shadow_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.3_1 2019-12-09T10:57:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__multipart_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.4 2019-12-09T10:57:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__shadow_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.4_1 2019-12-09T10:57:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.147+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.30__multipart_key1.2~9aIu0Y89uvJpaNjRckHB1-2l73HoDQ5.5 2019-12-09T10:57:53.380 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.374+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:53.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.454+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:57:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.489+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:53.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.505+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:53.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:53.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.460', time=2019-12-09T12:18:00.247241+0000, chain.objs.size()=7 2019-12-09T10:57:53.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.1 2019-12-09T10:57:53.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.1_1 2019-12-09T10:57:53.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.1_2 2019-12-09T10:57:53.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.2 2019-12-09T10:57:53.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.2_1 2019-12-09T10:57:53.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.2_2 2019-12-09T10:57:53.533 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.524+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~6sRIraWgWS8At4w8oE8Cb7BWrWzqnbH.3 2019-12-09T10:57:53.817 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.811+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:53.907 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.901+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:57:53.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.943+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.961+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:53.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:53.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.472', time=2019-12-09T12:18:33.821213+0000, chain.objs.size()=7 2019-12-09T10:57:53.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.1 2019-12-09T10:57:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.1_1 2019-12-09T10:57:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.1_2 2019-12-09T10:57:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.2 2019-12-09T10:57:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.2_1 2019-12-09T10:57:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__shadow_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.2_2 2019-12-09T10:57:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:53.973+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.20__multipart_key1.2~2qybzo_paj2WZFMJgJ_Dq1gRZKdkpXM.3 2019-12-09T10:57:54.186 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.180+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:54.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.313+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:57:54.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.361+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:54.384 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.377+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.387+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.426+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:57:54.494 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.488+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:54.516 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.511+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:54.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.386', time=2019-12-09T12:12:23.019446+0000, chain.objs.size()=9 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__multipart_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.1 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__shadow_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.1_1 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__multipart_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.2 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__shadow_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.2_1 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__multipart_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.3 2019-12-09T10:57:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__shadow_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.3_1 2019-12-09T10:57:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__multipart_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.4 2019-12-09T10:57:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__shadow_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.4_1 2019-12-09T10:57:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.531+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.15__multipart_key1.2~Q8fLStZHWu2czQDTkB7ZIX0WTqzaRWZ.5 2019-12-09T10:57:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.809+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:54.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.935+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:57:54.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:54.985+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:55.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.005+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.021+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.021+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.553', time=2019-12-09T12:22:23.835059+0000, chain.objs.size()=9 2019-12-09T10:57:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.021+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__multipart_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.1 2019-12-09T10:57:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__shadow_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.1_1 2019-12-09T10:57:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__multipart_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.2 2019-12-09T10:57:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__shadow_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.2_1 2019-12-09T10:57:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__multipart_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.3 2019-12-09T10:57:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__shadow_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.3_1 2019-12-09T10:57:55.031 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__multipart_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.4 2019-12-09T10:57:55.031 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__shadow_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.4_1 2019-12-09T10:57:55.031 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.022+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.26__multipart_file.mpg.2~GmleYMqVk7h_5hFjdP27MI53PYechx1.5 2019-12-09T10:57:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.281+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:55.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.382+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:57:55.443 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.421+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.439+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:55.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.450+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:55.486 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.480+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:57:55.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.544+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.566+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:55.591 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.583+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:55.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.640+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:57:55.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.683+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.699+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:57:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.528', time=2019-12-09T12:21:13.888642+0000, chain.objs.size()=9 2019-12-09T10:57:55.720 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__multipart_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.1 2019-12-09T10:57:55.720 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__shadow_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.1_1 2019-12-09T10:57:55.720 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__multipart_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.2 2019-12-09T10:57:55.720 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__shadow_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.2_1 2019-12-09T10:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__multipart_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.3 2019-12-09T10:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__shadow_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.3_1 2019-12-09T10:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__multipart_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.4 2019-12-09T10:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__shadow_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.4_1 2019-12-09T10:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.712+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.25__multipart_key1.2~rA38yQ8Qsh-DCQaLqw3jUMsAbeJ5fYl.5 2019-12-09T10:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:55.998+0000 7f45f35c9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:57:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.150+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:57:56.231 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.225+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:56.261 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.255+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:56.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.271+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:56.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.318+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:57:56.377 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.372+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:56.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.391+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:56.405 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.399+0000 7f45f35c9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.511+0000 7f45f35c9ec0 20 remove_watcher() i=0 2019-12-09T10:57:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.511+0000 7f45f35c9ec0 2 removed watcher, disabling cache 2019-12-09T10:57:56.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.573+0000 7f45f35c9ec0 20 remove_watcher() i=1 2019-12-09T10:57:56.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.615+0000 7f45f35c9ec0 20 remove_watcher() i=2 2019-12-09T10:57:56.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.682+0000 7f45f35c9ec0 20 remove_watcher() i=3 2019-12-09T10:57:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.721+0000 7f45f35c9ec0 20 remove_watcher() i=4 2019-12-09T10:57:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.760+0000 7f45f35c9ec0 20 remove_watcher() i=5 2019-12-09T10:57:56.829 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.823+0000 7f45f35c9ec0 20 remove_watcher() i=6 2019-12-09T10:57:56.881 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:56.875+0000 7f45f35c9ec0 20 remove_watcher() i=7 2019-12-09T10:57:56.891 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:57:57.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.202+0000 7fcd5aa44700 0 --2- 172.21.15.59:0/707380186 >> v2:172.21.15.59:3300/0 conn(0x55f91bac2a50 0x55f91bb9ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:57:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.241+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.283 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.277+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.283 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.278+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.288+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.288+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.299+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.299+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.328+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:57.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.525+0000 7fcd6e01bec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:57:57.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.525+0000 7fcd6e01bec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:57:57.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.525+0000 7fcd6e01bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:57:57.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.525+0000 7fcd6e01bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:57:57.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.525+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.543 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.537+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:57.543 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.537+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.559+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:57:57.565 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.559+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.582 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.575+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:57.582 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.575+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.589 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.582+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:57:57.589 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.582+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.593+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.593+0000 7fcd6e01bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:57:57.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.593+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.601+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.601+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.613+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:57.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.613+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.624+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:57:57.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.624+0000 7fcd6e01bec0 20 zonegroup default 2019-12-09T10:57:57.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.624+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.633+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.633+0000 7fcd6e01bec0 10 Cannot find current period zone using local zone 2019-12-09T10:57:57.639 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.633+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.642+0000 7fcd6e01bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:57:57.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.642+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.654+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:57:57.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.654+0000 7fcd6e01bec0 20 rados->read ofs=0 len=0 2019-12-09T10:57:57.672 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.667+0000 7fcd6e01bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:57:57.673 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.667+0000 7fcd6e01bec0 20 zone default found 2019-12-09T10:57:57.673 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.667+0000 7fcd6e01bec0 20 started sync module instance, tier type = 2019-12-09T10:57:57.673 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.667+0000 7fcd6e01bec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:57:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:57.993+0000 7fcd5b245700 0 -- 172.21.15.59:0/1816130989 >> [v2:172.21.15.59:6818/9835,v1:172.21.15.59:6819/9835] conn(0x7fcd5400a550 msgr2=0x7fcd5400c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:57:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=0 2019-12-09T10:57:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=1 2019-12-09T10:57:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=2 2019-12-09T10:57:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=3 2019-12-09T10:57:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=4 2019-12-09T10:57:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=5 2019-12-09T10:57:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.000+0000 7fcd6e01bec0 20 add_watcher() i=6 2019-12-09T10:57:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.033+0000 7fcd6e01bec0 20 add_watcher() i=7 2019-12-09T10:57:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.033+0000 7fcd6e01bec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:57:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.034+0000 7fcd2bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:57:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:58.050+0000 7fcd5b245700 0 -- 172.21.15.59:0/1183781885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f91bbc6410 msgr2=0x55f91bbc6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:57:59.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.712+0000 7fcc9a7fc700 20 reqs_thread_entry: start 2019-12-09T10:57:59.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.712+0000 7fcc93fff700 20 reqs_thread_entry: start 2019-12-09T10:57:59.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.713+0000 7fcd6e01bec0 20 init_complete bucket index max shards: 0 2019-12-09T10:57:59.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.713+0000 7fcc927fc700 20 reqs_thread_entry: start 2019-12-09T10:57:59.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.713+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:57:59.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.762+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:59.786 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.781+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:59.797 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.791+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.831+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:57:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.899+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:57:59.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.922+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:57:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.935+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:57:59.996+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:58:00.077 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.071+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.099+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:00.126 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.116+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.160+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:58:00.237 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.231+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.273 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.262+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.280+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.346+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:58:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.408+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.436 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.430+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:00.447 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.442+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.491 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.486+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:58:00.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.592+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.644+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.661+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.745+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:58:00.826 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.821+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.847 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.842+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:00.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.854+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:00.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.918+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:58:00.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.968+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:00.999+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.013+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.057 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.051+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:58:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.110+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.164+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.173+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.222+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:58:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.299+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.330 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.324+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.343+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.390+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:58:01.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.449+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.478 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.472+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.487 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.481+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.533 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.527+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:58:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.597+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.631 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.625+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.639+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.688+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:58:01.749 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.743+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.761+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.773+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.815+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:58:01.873 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.867+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.887+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.896+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:01.938+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:58:02.014 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.008+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.058+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.086+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.149 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.143+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:58:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.191+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.216 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.210+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.221+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.276 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.270+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:58:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.314+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.331+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.340+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.378 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.372+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:58:02.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.415+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.429+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.449 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.444+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.479+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:58:02.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.519+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.541 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.535+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.544+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.582 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.576+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:58:02.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.635+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.667 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.660+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.673+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.716+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:58:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.763+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.792 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.787+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.803 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.798+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.849 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.843+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:58:02.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.907+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.925+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.935+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:02.983 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:02.977+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:58:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.053+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.083+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.100 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.095+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.131+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:58:03.186 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.180+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.203 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.197+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.206+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.237+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:58:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.312+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.338 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.332+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.344+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.386+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:58:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.438+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.469 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.464+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.477+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.517+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:58:03.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.581+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.600+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.609+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.644+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:58:03.704 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.698+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.723 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.717+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.729+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.766+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:58:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.828+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.852+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.869 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.863+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.899 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.894+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:58:03.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.941+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.961+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:03.977 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:03.971+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:04.015 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.009+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:58:04.068 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.063+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.083+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:04.099 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.093+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.135+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:58:04.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.283+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:04.317 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.312+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.328+0000 7fcd6e01bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:04.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.440+0000 7fcd6e01bec0 20 remove_watcher() i=0 2019-12-09T10:58:04.446 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.440+0000 7fcd6e01bec0 2 removed watcher, disabling cache 2019-12-09T10:58:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.503+0000 7fcd6e01bec0 20 remove_watcher() i=1 2019-12-09T10:58:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.567+0000 7fcd6e01bec0 20 remove_watcher() i=2 2019-12-09T10:58:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.652+0000 7fcd6e01bec0 20 remove_watcher() i=3 2019-12-09T10:58:04.699 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.694+0000 7fcd6e01bec0 20 remove_watcher() i=4 2019-12-09T10:58:04.749 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.743+0000 7fcd6e01bec0 20 remove_watcher() i=5 2019-12-09T10:58:04.796 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.790+0000 7fcd6e01bec0 20 remove_watcher() i=6 2019-12-09T10:58:04.827 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.821+0000 7fcd6e01bec0 20 remove_watcher() i=7 2019-12-09T10:58:04.833 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:58:04.939 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.933+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:04.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.965+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:04.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.966+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:04.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.981+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:04.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.981+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.989+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:04.989+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.016 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.010+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.217+0000 7f8cca8dbec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:58:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.217+0000 7f8cca8dbec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:58:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.217+0000 7f8cca8dbec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:58:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.217+0000 7f8cca8dbec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:58:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.217+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.238 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.232+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:05.238 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.232+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.248+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:58:05.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.248+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.280 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.274+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:05.280 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.274+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.289+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:58:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.289+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.303 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.298+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:05.303 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.298+0000 7f8cca8dbec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:58:05.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.298+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.309 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.304+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:05.309 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.304+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.321+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.321+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.339+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:58:05.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.339+0000 7f8cca8dbec0 20 zonegroup default 2019-12-09T10:58:05.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.339+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.351+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:05.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.351+0000 7f8cca8dbec0 10 Cannot find current period zone using local zone 2019-12-09T10:58:05.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.351+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.362+0000 7f8cca8dbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:05.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.362+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.386 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.381+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:05.386 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.381+0000 7f8cca8dbec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.398+0000 7f8cca8dbec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:58:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.398+0000 7f8cca8dbec0 20 zone default found 2019-12-09T10:58:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.398+0000 7f8cca8dbec0 20 started sync module instance, tier type = 2019-12-09T10:58:05.407 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.398+0000 7f8cca8dbec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:58:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.785+0000 7f8cca8dbec0 20 add_watcher() i=0 2019-12-09T10:58:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.785+0000 7f8cca8dbec0 20 add_watcher() i=1 2019-12-09T10:58:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.785+0000 7f8cca8dbec0 20 add_watcher() i=2 2019-12-09T10:58:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.785+0000 7f8cca8dbec0 20 add_watcher() i=3 2019-12-09T10:58:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.785+0000 7f8cca8dbec0 20 add_watcher() i=4 2019-12-09T10:58:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.785+0000 7f8cca8dbec0 20 add_watcher() i=5 2019-12-09T10:58:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.787+0000 7f8cca8dbec0 20 add_watcher() i=6 2019-12-09T10:58:05.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.813+0000 7f8cca8dbec0 20 add_watcher() i=7 2019-12-09T10:58:05.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.813+0000 7f8cca8dbec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:58:05.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.813+0000 7f8c0bfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:58:05.842 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.836+0000 7f8cb8306700 0 -- 172.21.15.59:0/2384079967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf8002830 msgr2=0x7f8bf8002e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:58:05.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:05.882+0000 7f8cb7304700 0 -- 172.21.15.59:0/675430703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583f7924810 msgr2=0x5583f7926c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:58:07.640 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.634+0000 7f8bf67fc700 20 reqs_thread_entry: start 2019-12-09T10:58:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.635+0000 7f8beffff700 20 reqs_thread_entry: start 2019-12-09T10:58:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.635+0000 7f8cca8dbec0 20 init_complete bucket index max shards: 0 2019-12-09T10:58:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.636+0000 7f8bee7fc700 20 reqs_thread_entry: start 2019-12-09T10:58:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.636+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:58:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.708+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:07.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.736+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.746+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:07.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.801+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:58:07.875 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.870+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:07.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.889+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.898+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:07.943+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:58:08.007 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.001+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.027+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.047 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.041+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.086+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:58:08.183 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.177+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.208+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.236 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.227+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.307 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.301+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:58:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.372+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.396+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.407+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.440+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:58:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.489+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.507+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.517+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.556 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.551+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:58:08.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.597+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.620 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.614+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.623+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.662+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:58:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.706+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.722+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.730+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.764+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:58:08.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.807+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.825+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.835+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.871+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:58:08.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.922+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.939+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:08.954 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.949+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:08.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:08.982+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:58:09.045 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.039+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.074 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.068+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:09.085 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.079+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.131 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.125+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:58:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.180+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.205+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.217+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.265+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:58:09.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.368+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.390+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.405+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.450+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:58:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.537+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.577 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.571+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:09.596 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.590+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.637+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:58:09.727 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.721+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.760+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.785+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.834+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:58:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.890+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.909+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:09.924 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.918+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:09.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:09.952+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:58:10.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.002+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.025+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.034+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.071 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.065+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:58:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.112+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.132+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.144+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.190+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:58:10.268 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.262+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.292+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.319+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.378 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.372+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:58:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.444+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.472+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.487+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.542 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.536+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:58:10.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.600+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.622+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.633+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.672+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:58:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.730+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.752+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.764+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.810+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:58:10.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.881+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.905+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:10.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.921+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:10.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:10.957+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:58:11.023 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.017+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.039+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.067 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.061+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.122 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.116+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:58:11.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.174+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.200 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.193+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.208+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.261 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.255+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:58:11.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.323+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.355 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.349+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.366+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.412+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:58:11.485 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.470+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.499 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.493+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.503+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.554+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:58:11.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.638+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.671 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.666+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.684 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.678+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.738+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:58:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.802+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.832 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.827+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.844 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.839+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.888+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:58:11.939 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.933+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.955 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.950+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:11.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.959+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:11.998 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:11.992+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:58:12.045 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.040+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:12.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.066+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.076+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:12.115 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.109+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:58:12.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.157+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:12.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.176+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:12.190 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.185+0000 7f8cca8dbec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:12.285 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.280+0000 7f8cca8dbec0 20 remove_watcher() i=0 2019-12-09T10:58:12.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.280+0000 7f8cca8dbec0 2 removed watcher, disabling cache 2019-12-09T10:58:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.345+0000 7f8cca8dbec0 20 remove_watcher() i=1 2019-12-09T10:58:12.404 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.398+0000 7f8cca8dbec0 20 remove_watcher() i=2 2019-12-09T10:58:12.482 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.471+0000 7f8cca8dbec0 20 remove_watcher() i=3 2019-12-09T10:58:12.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.520+0000 7f8cca8dbec0 20 remove_watcher() i=4 2019-12-09T10:58:12.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.572+0000 7f8cca8dbec0 20 remove_watcher() i=5 2019-12-09T10:58:12.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.640+0000 7f8cca8dbec0 20 remove_watcher() i=6 2019-12-09T10:58:12.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.688+0000 7f8cca8dbec0 20 remove_watcher() i=7 2019-12-09T10:58:12.703 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:58:12.847 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.841+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:12.898 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.892+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:12.898 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.892+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:12.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.905+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:12.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.905+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:12.919 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.914+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:12.920 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.914+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:12.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:12.935+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:13.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.169+0000 7fd663253ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:58:13.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.169+0000 7fd663253ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:58:13.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.169+0000 7fd663253ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:58:13.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.169+0000 7fd663253ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:58:13.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.169+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.188 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.183+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:13.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.183+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.199+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:58:13.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.199+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.217 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.211+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:13.217 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.211+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.223+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:58:13.229 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.223+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.235+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.235+0000 7fd663253ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.235+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.245+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.245+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.257+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.257+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.273+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:58:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.273+0000 7fd663253ec0 20 zonegroup default 2019-12-09T10:58:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.273+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.282+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.282+0000 7fd663253ec0 10 Cannot find current period zone using local zone 2019-12-09T10:58:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.282+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.288+0000 7fd663253ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:58:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.288+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.301+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:58:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.301+0000 7fd663253ec0 20 rados->read ofs=0 len=0 2019-12-09T10:58:13.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.317+0000 7fd663253ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:58:13.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.317+0000 7fd663253ec0 20 zone default found 2019-12-09T10:58:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.317+0000 7fd663253ec0 20 started sync module instance, tier type = 2019-12-09T10:58:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.317+0000 7fd663253ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:58:13.717 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=0 2019-12-09T10:58:13.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=1 2019-12-09T10:58:13.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=2 2019-12-09T10:58:13.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=3 2019-12-09T10:58:13.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=4 2019-12-09T10:58:13.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=5 2019-12-09T10:58:13.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.712+0000 7fd663253ec0 20 add_watcher() i=6 2019-12-09T10:58:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.748+0000 7fd663253ec0 20 add_watcher() i=7 2019-12-09T10:58:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.748+0000 7fd663253ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:58:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.748+0000 7fd620fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:58:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.811+0000 7fd64fc7c700 0 -- 172.21.15.59:0/3896055380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a41de8a780 msgr2=0x55a41de8ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:58:13.916 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:13.910+0000 7fd650c7e700 0 -- 172.21.15.59:0/2969828304 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7fd594022470 msgr2=0x7fd594024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:58:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.861+0000 7fd592ffd700 20 reqs_thread_entry: start 2019-12-09T10:58:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.861+0000 7fd590ff9700 20 reqs_thread_entry: start 2019-12-09T10:58:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.861+0000 7fd663253ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:58:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.862+0000 7fd57effd700 20 reqs_thread_entry: start 2019-12-09T10:58:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.862+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:58:15.920 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.914+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:15.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.934+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:15.951 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.945+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:15.993+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:58:16.066 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.060+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.089+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:16.118 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.112+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.168+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:58:16.261 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.255+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.292+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:16.316 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.310+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.379 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.373+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:58:16.452 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.446+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.482 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.476+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:16.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.492+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.554 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.548+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:58:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.603+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.627+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.640+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.684 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.679+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:58:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.788+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.816 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.810+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.826+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.877+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:58:16.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.958+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:16.999 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:16.993+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.021 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.016+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.059+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:58:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.119+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.159+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.173+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.245 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.239+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:58:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.282+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.300+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.317 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.312+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.346+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:58:17.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.385+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.402+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.411+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.451 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.445+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:58:17.492 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.486+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.505+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.520 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.514+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.555 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.549+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:58:17.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.597+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.615+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.627+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.656+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:58:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.705+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.723+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.734+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.778 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.772+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:58:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.831+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.873 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.867+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.876+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.908+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:58:17.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.952+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:17.976 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.970+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:17.984 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:17.979+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.023+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:58:18.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.122+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.150+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:18.169 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.164+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.217+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:58:18.275 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.269+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.293+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.306+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.343+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:58:18.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.412+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.435 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.429+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.438+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.478+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:58:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.526+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.546+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.555+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.611+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:58:18.678 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.672+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.711+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:18.735 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.731+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.775 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.771+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:58:18.853 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.848+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.871 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.867+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:18.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.878+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.922 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.917+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:58:18.977 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.972+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:18.991+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.000+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.033+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.087+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.107+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.119+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.163 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.158+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:58:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.217+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.241 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.236+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.250+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.292 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.287+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:58:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.355+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.378 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.374+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.387 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.382+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.417+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:58:19.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.479+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.497+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.509+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.550+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:58:19.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.649+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.689+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.738 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.734+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.802+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:58:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.878+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.908+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:19.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.919+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:19.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:19.958+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:58:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.005+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.028 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.023+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:20.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.035+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.074+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:58:20.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.135+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.155+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:20.173 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.169+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.209+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:58:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.275+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.297+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:20.313 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.308+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.356 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.352+0000 7fd663253ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:58:20.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.416+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.448 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.444+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:58:20.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.456+0000 7fd663253ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:58:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.559+0000 7fd663253ec0 20 remove_watcher() i=0 2019-12-09T10:58:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.559+0000 7fd663253ec0 2 removed watcher, disabling cache 2019-12-09T10:58:20.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.621+0000 7fd663253ec0 20 remove_watcher() i=1 2019-12-09T10:58:20.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.678+0000 7fd663253ec0 20 remove_watcher() i=2 2019-12-09T10:58:20.725 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.721+0000 7fd663253ec0 20 remove_watcher() i=3 2019-12-09T10:58:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.775+0000 7fd663253ec0 20 remove_watcher() i=4 2019-12-09T10:58:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.827+0000 7fd663253ec0 20 remove_watcher() i=5 2019-12-09T10:58:20.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.888+0000 7fd663253ec0 20 remove_watcher() i=6 2019-12-09T10:58:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:58:20.933+0000 7fd663253ec0 20 remove_watcher() i=7 2019-12-09T10:58:20.946 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-12-09T10:58:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :clean 2019-12-09T10:58:22.167 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :compileJava 2019-12-09T10:58:22.167 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :processResources 2019-12-09T10:58:22.167 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :classes 2019-12-09T10:58:22.468 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:22.469 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :compileTestJava 2019-12-09T10:58:22.469 INFO:teuthology.orchestra.run.smithi059.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T10:58:22.469 INFO:teuthology.orchestra.run.smithi059.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T10:58:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:22.470 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T10:58:22.470 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :testClasses 2019-12-09T10:58:25.366 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:25.367 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :test 2019-12-09T10:58:25.367 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:25.367 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-12-09T10:58:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:27.766 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-12-09T10:58:35.066 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:35.066 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-12-09T10:58:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:37.166 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-12-09T10:58:43.966 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:43.966 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-12-09T10:58:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:47.166 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-12-09T10:58:53.566 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:58:53.566 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-12-09T10:59:00.866 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:00.866 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-12-09T10:59:02.766 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:02.766 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-12-09T10:59:07.266 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:07.266 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-12-09T10:59:09.012 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:16:05.630 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.056 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:16:05.674 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.071 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:16:05.689 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.074 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:16:05.691 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.101 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:16:05.700 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.105 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:16:05.704 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.150 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:16:05.767 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.179 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:16:05.797 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:09.196 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:16:05.795 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T10:59:15.766 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:15.767 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-12-09T10:59:15.966 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:15.966 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-12-09T10:59:17.766 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:17.766 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-12-09T10:59:24.166 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:24.166 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-12-09T10:59:38.166 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:38.166 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-12-09T10:59:41.169 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:41.169 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-12-09T10:59:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:50.666 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-12-09T10:59:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:51.366 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-12-09T10:59:52.495 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:52.495 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------- 2019-12-09T10:59:52.495 INFO:teuthology.orchestra.run.smithi059.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-12-09T10:59:52.496 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------- 2019-12-09T10:59:52.496 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T10:59:52.496 INFO:teuthology.orchestra.run.smithi059.stdout:BUILD SUCCESSFUL in 1m 31s 2019-12-09T10:59:52.496 INFO:teuthology.orchestra.run.smithi059.stdout:5 actionable tasks: 5 executed 2019-12-09T10:59:52.501 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T10:59:52.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.627+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.663+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.663+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.672+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.672+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.681+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.681+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.703 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.699+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:59:52.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.921+0000 7f468ec53ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T10:59:52.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.921+0000 7f468ec53ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T10:59:52.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.921+0000 7f468ec53ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:59:52.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.921+0000 7f468ec53ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:59:52.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.921+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.955 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.951+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:59:52.956 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.951+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.966+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:59:52.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.967+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:52.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.984+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:59:52.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.984+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.999+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:59:53.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:52.999+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.005+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:53.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.005+0000 7f468ec53ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:59:53.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.005+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.015+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.015+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.025+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:59:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.025+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.041 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.037+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:59:53.041 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.037+0000 7f468ec53ec0 20 zonegroup default 2019-12-09T10:59:53.041 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.037+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.045+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:53.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.045+0000 7f468ec53ec0 10 Cannot find current period zone using local zone 2019-12-09T10:59:53.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.045+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.050+0000 7f468ec53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:59:53.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.050+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.080 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.076+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:59:53.081 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.076+0000 7f468ec53ec0 20 rados->read ofs=0 len=0 2019-12-09T10:59:53.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.088+0000 7f468ec53ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:59:53.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.088+0000 7f468ec53ec0 20 zone default found 2019-12-09T10:59:53.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.088+0000 7f468ec53ec0 20 started sync module instance, tier type = 2019-12-09T10:59:53.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.088+0000 7f468ec53ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T10:59:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=0 2019-12-09T10:59:53.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=1 2019-12-09T10:59:53.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=2 2019-12-09T10:59:53.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=3 2019-12-09T10:59:53.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=4 2019-12-09T10:59:53.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=5 2019-12-09T10:59:53.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.472+0000 7f468ec53ec0 20 add_watcher() i=6 2019-12-09T10:59:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.502+0000 7f468ec53ec0 20 add_watcher() i=7 2019-12-09T10:59:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.502+0000 7f468ec53ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:59:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.503+0000 7f45cffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:59:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.509+0000 7f467c67e700 0 -- 172.21.15.59:0/546117706 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f2a745b3e0 msgr2=0x55f2a7458c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:59:53.716 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.712+0000 7f467c67e700 0 --2- 172.21.15.59:0/546117706 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f2a745b3e0 0x55f2a7458c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:59:53.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:53.800+0000 7f467b67c700 0 -- 172.21.15.59:0/854901496 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x55f2a748df40 msgr2=0x55f2a7490360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:59:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.475+0000 7f45ba7fc700 20 reqs_thread_entry: start 2019-12-09T10:59:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.475+0000 7f45b3fff700 20 reqs_thread_entry: start 2019-12-09T10:59:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.475+0000 7f468ec53ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:59:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.476+0000 7f45b27fc700 20 reqs_thread_entry: start 2019-12-09T10:59:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.476+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:59:55.536 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.531+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:55.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.572+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:55.594 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.589+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:55.684 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.679+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:59:55.762 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.758+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:55.795 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.791+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:55.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.803+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:55.846 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.841+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:59:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.909+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:55.938 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.934+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:55.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.945+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:55.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:55.985+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:59:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.032+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.063+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:56.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.074+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.133+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:59:56.220 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.215+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.242+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.254+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.310 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.305+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:59:56.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.384+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.416+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:56.440 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.435+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.492 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.487+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:59:56.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.556+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.590 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.585+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:56.602 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.598+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.648+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:59:56.741 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.736+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.760+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.777+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.852 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.847+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:59:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.917+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.943+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:56.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.955+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:56.991 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:56.987+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:59:57.079 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.074+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.127 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.122+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:57.158 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.154+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.218+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:59:57.290 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.286+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.309+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.327+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.380+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:59:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.453+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.476+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.490+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.533+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:59:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.582+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.604 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.600+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.609+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.663+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:59:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.702+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.724 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.720+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.728+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.762+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:59:57.852 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.848+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.889 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.884+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:57.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.902+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:57.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:57.952+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:59:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.029+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.051+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:58.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.058+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.118 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.114+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:59:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.177+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.208 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.203+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:58.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.216+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.272 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.268+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:59:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.361+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.383+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:58.400 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.395+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.444 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.439+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:59:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.513+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.547 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.543+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.559+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.753 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.748+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:59:58.811 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.807+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.831+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:58.847 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.842+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.885+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:59:58.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.942+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:58.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.977+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:58.995+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.043 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.038+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:59:59.126 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.121+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.140+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.150+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.191+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:59:59.239 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.234+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.253+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.263+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.297+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:59:59.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.357+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.384 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.380+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.391+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.441+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:59:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.501+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.526+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.546 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.542+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.582+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:59:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.654+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.690 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.685+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.701+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.752+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:59:59.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.835+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.871 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.866+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:59:59.886 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.882+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:59:59.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T10:59:59.946+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T11:00:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.005+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.030+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:00.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.044+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.086+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T11:00:00.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.169+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.202+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.229+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.302+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T11:00:00.379 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.374+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.394+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:00.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.413+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.471 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.466+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T11:00:00.544 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.540+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.577 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.572+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.588+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.677+0000 7f468ec53ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T11:00:00.772 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.767+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.800+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.817+0000 7f468ec53ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.929+0000 7f468ec53ec0 20 remove_watcher() i=0 2019-12-09T11:00:00.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.929+0000 7f468ec53ec0 2 removed watcher, disabling cache 2019-12-09T11:00:00.998 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:00.993+0000 7f468ec53ec0 20 remove_watcher() i=1 2019-12-09T11:00:01.051 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.047+0000 7f468ec53ec0 20 remove_watcher() i=2 2019-12-09T11:00:01.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.087+0000 7f468ec53ec0 20 remove_watcher() i=3 2019-12-09T11:00:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.142+0000 7f468ec53ec0 20 remove_watcher() i=4 2019-12-09T11:00:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.198+0000 7f468ec53ec0 20 remove_watcher() i=5 2019-12-09T11:00:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.265+0000 7f468ec53ec0 20 remove_watcher() i=6 2019-12-09T11:00:01.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.317+0000 7f468ec53ec0 20 remove_watcher() i=7 2019-12-09T11:00:01.330 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T11:00:01.450 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.445+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.484 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.476+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.484 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.477+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.485+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.485+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.500 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.496+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.496+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.512+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:01.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.706+0000 7f3a1c91dec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T11:00:01.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.706+0000 7f3a1c91dec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T11:00:01.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.706+0000 7f3a1c91dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:00:01.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.706+0000 7f3a1c91dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:00:01.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.706+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.714+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:01.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.714+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.729+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:00:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.729+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.748 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.743+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:01.748 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.743+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.763 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.759+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:00:01.764 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.759+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.770+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.770+0000 7f3a1c91dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:00:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.770+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.777+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.777+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.789+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:01.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.789+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.799+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:00:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.799+0000 7f3a1c91dec0 20 zonegroup default 2019-12-09T11:00:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.799+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.816+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.816+0000 7f3a1c91dec0 10 Cannot find current period zone using local zone 2019-12-09T11:00:01.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.816+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.835 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.830+0000 7f3a1c91dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:01.835 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.830+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.851 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.847+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:01.851 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.847+0000 7f3a1c91dec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:01.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.863+0000 7f3a1c91dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:00:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.863+0000 7f3a1c91dec0 20 zone default found 2019-12-09T11:00:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.863+0000 7f3a1c91dec0 20 started sync module instance, tier type = 2019-12-09T11:00:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:01.863+0000 7f3a1c91dec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T11:00:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.210+0000 7f3a1c91dec0 20 add_watcher() i=0 2019-12-09T11:00:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.210+0000 7f3a1c91dec0 20 add_watcher() i=1 2019-12-09T11:00:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.210+0000 7f3a1c91dec0 20 add_watcher() i=2 2019-12-09T11:00:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.210+0000 7f3a1c91dec0 20 add_watcher() i=3 2019-12-09T11:00:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.210+0000 7f3a1c91dec0 20 add_watcher() i=4 2019-12-09T11:00:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.210+0000 7f3a1c91dec0 20 add_watcher() i=5 2019-12-09T11:00:02.217 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.213+0000 7f3a1c91dec0 20 add_watcher() i=6 2019-12-09T11:00:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.248+0000 7f3a1c91dec0 20 add_watcher() i=7 2019-12-09T11:00:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.248+0000 7f3a1c91dec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:00:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.248+0000 7f39da7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:00:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.251+0000 7f3a09b47700 0 -- 172.21.15.59:0/2253508689 >> v2:172.21.15.59:3300/0 conn(0x55869619e660 msgr2=0x55869619aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T11:00:02.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:02.289+0000 7f3a09346700 0 -- 172.21.15.59:0/1046941232 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x5586961c7490 msgr2=0x5586961c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T11:00:04.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.045+0000 7f3950ff9700 20 reqs_thread_entry: start 2019-12-09T11:00:04.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.045+0000 7f394a7fc700 20 reqs_thread_entry: start 2019-12-09T11:00:04.051 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.045+0000 7f3a1c91dec0 20 init_complete bucket index max shards: 0 2019-12-09T11:00:04.051 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.046+0000 7f3948ff9700 20 reqs_thread_entry: start 2019-12-09T11:00:04.052 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.046+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T11:00:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.113+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.149+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.169+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.219+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T11:00:04.274 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.270+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.295+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:04.315 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.310+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.353+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T11:00:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.431+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.455+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.466+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.512 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.507+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T11:00:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.576+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.602 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.598+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:04.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.611+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.652+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T11:00:04.717 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.712+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.736+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.753+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.803 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.798+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T11:00:04.892 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.888+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:04.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.923+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:04.942+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.009 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.004+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T11:00:05.066 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.061+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.090 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.085+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:05.104 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.100+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.144+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T11:00:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.201+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.222+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:05.238 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.233+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.271+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T11:00:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.321+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.341 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.336+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:05.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.347+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.384+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T11:00:05.442 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.437+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.464 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.459+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.472+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.526+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T11:00:05.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.614+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.646+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:05.672 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.661+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.714+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T11:00:05.796 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.791+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.823+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:05.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.834+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:05.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.883+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T11:00:05.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:05.964+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.014 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.009+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:06.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.025+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.077+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T11:00:06.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.148+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.192+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.210+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.253+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T11:00:06.316 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.312+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.334 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.330+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:06.344 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.339+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.387+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T11:00:06.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.485+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.522+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.538+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.575+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T11:00:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.634+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.652+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.663+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.709+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T11:00:06.798 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.794+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.827+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:06.853 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.848+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.895+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T11:00:06.975 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.970+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:06.998 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:06.993+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.006+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.045+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T11:00:07.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.132+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.165+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.179+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.237 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.232+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T11:00:07.300 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.296+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.321+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.335+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.408+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T11:00:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.467+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.486+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.505 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.494+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.537 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.533+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T11:00:07.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.605+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.629+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.643+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.691+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T11:00:07.754 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.749+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.783 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.779+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.792+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.827+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T11:00:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.896+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.919+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:07.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.930+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:07.977 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:07.973+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T11:00:08.031 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.026+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.044+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.054+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.105 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.100+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T11:00:08.180 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.176+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.211 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.206+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.223+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.284 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.280+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T11:00:08.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.342+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.373 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.369+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.382 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.378+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.430 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.425+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T11:00:08.486 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.477+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.502+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.518 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.513+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.553+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T11:00:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.624+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.642+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.652+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.693 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.689+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T11:00:08.748 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.743+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.768 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.764+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.783 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.779+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.816+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T11:00:08.862 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.857+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.877+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.886+0000 7f3a1c91dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:08.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.988+0000 7f3a1c91dec0 20 remove_watcher() i=0 2019-12-09T11:00:08.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:08.988+0000 7f3a1c91dec0 2 removed watcher, disabling cache 2019-12-09T11:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.031+0000 7f3a1c91dec0 20 remove_watcher() i=1 2019-12-09T11:00:09.073 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.069+0000 7f3a1c91dec0 20 remove_watcher() i=2 2019-12-09T11:00:09.129 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.124+0000 7f3a1c91dec0 20 remove_watcher() i=3 2019-12-09T11:00:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.157+0000 7f3a1c91dec0 20 remove_watcher() i=4 2019-12-09T11:00:09.200 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.196+0000 7f3a1c91dec0 20 remove_watcher() i=5 2019-12-09T11:00:09.234 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.230+0000 7f3a1c91dec0 20 remove_watcher() i=6 2019-12-09T11:00:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.269+0000 7f3a1c91dec0 20 remove_watcher() i=7 2019-12-09T11:00:09.281 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T11:00:09.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.392+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.412+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.412+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.417+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.417+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.426 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.422+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.426 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.422+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.438 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.434+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.557+0000 7f18a1641ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T11:00:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.557+0000 7f18a1641ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T11:00:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.557+0000 7f18a1641ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:00:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.557+0000 7f18a1641ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:00:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.557+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.570+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.570+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.583 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.578+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:00:09.583 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.579+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.589+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.589+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.604+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:00:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.604+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.614 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.610+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.610+0000 7f18a1641ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:00:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.610+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.623 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.619+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.619+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.632+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:09.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.632+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.644+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:00:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.644+0000 7f18a1641ec0 20 zonegroup default 2019-12-09T11:00:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.644+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.655+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.655+0000 7f18a1641ec0 10 Cannot find current period zone using local zone 2019-12-09T11:00:09.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.655+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.664+0000 7f18a1641ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:09.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.664+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.678+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:09.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.678+0000 7f18a1641ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:09.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.691+0000 7f18a1641ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:00:09.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.691+0000 7f18a1641ec0 20 zone default found 2019-12-09T11:00:09.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.691+0000 7f18a1641ec0 20 started sync module instance, tier type = 2019-12-09T11:00:09.696 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:09.691+0000 7f18a1641ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T11:00:10.085 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=0 2019-12-09T11:00:10.085 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=1 2019-12-09T11:00:10.085 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=2 2019-12-09T11:00:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=3 2019-12-09T11:00:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=4 2019-12-09T11:00:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=5 2019-12-09T11:00:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.080+0000 7f18a1641ec0 20 add_watcher() i=6 2019-12-09T11:00:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.118+0000 7f18a1641ec0 20 add_watcher() i=7 2019-12-09T11:00:10.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.118+0000 7f18a1641ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:00:10.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.118+0000 7f185f7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:00:10.131 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.127+0000 7f188f06c700 0 -- 172.21.15.59:0/3724021027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a348b980c0 msgr2=0x55a348b97450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T11:00:10.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:10.170+0000 7f188f06c700 0 -- 172.21.15.59:0/1049153773 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7f17d0022470 msgr2=0x7f17d0024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:00:11.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.490+0000 7f17cdffb700 20 reqs_thread_entry: start 2019-12-09T11:00:11.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.490+0000 7f17bf7fe700 20 reqs_thread_entry: start 2019-12-09T11:00:11.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.490+0000 7f18a1641ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:00:11.496 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.491+0000 7f17bdffb700 20 reqs_thread_entry: start 2019-12-09T11:00:11.496 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.491+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T11:00:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.563+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:11.594 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.589+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:11.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.608+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:11.676 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.671+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T11:00:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.736+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.764+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.777+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:11.833 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.828+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T11:00:11.880 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.876+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.893+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.901+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:11.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.936+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T11:00:12.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:11.999+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.026 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.022+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.037 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.033+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.068+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T11:00:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.114+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.136 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.131+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.144 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.140+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.183+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T11:00:12.258 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.253+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.280+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.298 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.294+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.346+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T11:00:12.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.415+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.444 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.439+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.449+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.509 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.504+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T11:00:12.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.555+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.579+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.594 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.590+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.623+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T11:00:12.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.673+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.694 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.690+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.702+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.751 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.746+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T11:00:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.803+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.824+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.834+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.876+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T11:00:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.953+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.972+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:12.987+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.034 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.029+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T11:00:13.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.111+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.149+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:13.176 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.172+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.237+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T11:00:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.301+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.323+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.334+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.378 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.374+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T11:00:13.440 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.436+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.466 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.461+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:13.479 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.474+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.522+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T11:00:13.604 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.600+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.633+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.663+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.722+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T11:00:13.775 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.771+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.821+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:13.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.835+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.879+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T11:00:13.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.922+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.940+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:13.954 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.950+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:13.990+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T11:00:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.051+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.073+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.085+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.133+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T11:00:14.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.207+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.242 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.238+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:14.256 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.252+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.301+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T11:00:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.347+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.364+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:14.383 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.379+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.426 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.422+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T11:00:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.469+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.498+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:14.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.509+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.563+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T11:00:14.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.713+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.751+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.772+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.830+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T11:00:14.907 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.903+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:14.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.930+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:14.942+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.001+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T11:00:15.087 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.083+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.124+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:15.149 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.144+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.240 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.236+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T11:00:15.313 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.308+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.342 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.338+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.353+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.419 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.414+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T11:00:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.510+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.545 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.541+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:15.556 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.551+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.606 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.602+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T11:00:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.686+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.716+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:15.735 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.731+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.793 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.785+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T11:00:15.850 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.845+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.863+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.875+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:15.914+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T11:00:16.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.044+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.060+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:16.074 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.070+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.129 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.121+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T11:00:16.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.202+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.233 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.229+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.243+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.290 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.286+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T11:00:16.402 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.393+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.427 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.423+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:16.443 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.438+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.494 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.489+0000 7f18a1641ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T11:00:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.558+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.584+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:16.602 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.597+0000 7f18a1641ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:16.699 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.694+0000 7f18a1641ec0 20 remove_watcher() i=0 2019-12-09T11:00:16.699 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.694+0000 7f18a1641ec0 2 removed watcher, disabling cache 2019-12-09T11:00:16.756 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.752+0000 7f18a1641ec0 20 remove_watcher() i=1 2019-12-09T11:00:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.817+0000 7f18a1641ec0 20 remove_watcher() i=2 2019-12-09T11:00:16.863 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.858+0000 7f18a1641ec0 20 remove_watcher() i=3 2019-12-09T11:00:16.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.899+0000 7f18a1641ec0 20 remove_watcher() i=4 2019-12-09T11:00:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:16.951+0000 7f18a1641ec0 20 remove_watcher() i=5 2019-12-09T11:00:17.022 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.017+0000 7f18a1641ec0 20 remove_watcher() i=6 2019-12-09T11:00:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.072+0000 7f18a1641ec0 20 remove_watcher() i=7 2019-12-09T11:00:17.087 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T11:00:17.242 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.237+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.281 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.277+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.281 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.277+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.286+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.286+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.297+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.297+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.324+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.592+0000 7f3c68154ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T11:00:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.592+0000 7f3c68154ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T11:00:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.592+0000 7f3c68154ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:00:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.592+0000 7f3c68154ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:00:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.592+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.606+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.606+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.619+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:00:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.619+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.636 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.632+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:17.636 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.632+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.652+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:00:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.652+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.658+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.658+0000 7f3c68154ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:00:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.658+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.665+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.665+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.675+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.675+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.707 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.703+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:00:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.703+0000 7f3c68154ec0 20 zonegroup default 2019-12-09T11:00:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.703+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.712+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.712+0000 7f3c68154ec0 10 Cannot find current period zone using local zone 2019-12-09T11:00:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.712+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.718+0000 7f3c68154ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:00:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.718+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.735 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.731+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:00:17.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.731+0000 7f3c68154ec0 20 rados->read ofs=0 len=0 2019-12-09T11:00:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.741+0000 7f3c68154ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:00:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.742+0000 7f3c68154ec0 20 zone default found 2019-12-09T11:00:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.742+0000 7f3c68154ec0 20 started sync module instance, tier type = 2019-12-09T11:00:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:17.742+0000 7f3c68154ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T11:00:18.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=0 2019-12-09T11:00:18.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=1 2019-12-09T11:00:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=2 2019-12-09T11:00:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=3 2019-12-09T11:00:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=4 2019-12-09T11:00:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=5 2019-12-09T11:00:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.155+0000 7f3c68154ec0 20 add_watcher() i=6 2019-12-09T11:00:18.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.188+0000 7f3c68154ec0 20 add_watcher() i=7 2019-12-09T11:00:18.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.188+0000 7f3c68154ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:00:18.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.188+0000 7f3c25fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:00:18.197 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.192+0000 7f3c4ffff700 0 -- 172.21.15.59:0/3788382980 >> v2:172.21.15.161:3300/0 conn(0x55c811dc1cc0 msgr2=0x55c811dc20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T11:00:18.244 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:18.240+0000 7f3c5537e700 0 -- 172.21.15.59:0/2483001225 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7f3b980252a0 msgr2=0x7f3b98027730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:00:19.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:19.984+0000 7f3b97fff700 20 reqs_thread_entry: start 2019-12-09T11:00:19.989 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:19.985+0000 7f3b95ffb700 20 reqs_thread_entry: start 2019-12-09T11:00:19.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:19.985+0000 7f3c68154ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:00:19.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:19.985+0000 7f3b83fff700 20 reqs_thread_entry: start 2019-12-09T11:00:19.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:19.985+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T11:00:20.046 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.041+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.062 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.058+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.071 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.067+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.103 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.098+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T11:00:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.182+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.210+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.219+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.274 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.270+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T11:00:20.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.341+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.367 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.362+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.377 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.372+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.416 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.412+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T11:00:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.476+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.498+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.512+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.556+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T11:00:20.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.606+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.629 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.624+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.633+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.659+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T11:00:20.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.700+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.716+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.725+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.756+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T11:00:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.796+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.818 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.813+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.822+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.850+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T11:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.934+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.961+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:20.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:20.975+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.029+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T11:00:21.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.085+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.107 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.101+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.115 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.110+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.151+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T11:00:21.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.207+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.228+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.246 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.240+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.298 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.293+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T11:00:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.359+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.398+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.428 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.423+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.478 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.472+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T11:00:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.522+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.543 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.538+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.553 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.547+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.592+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T11:00:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.658+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.689 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.684+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.702 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.697+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.747 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.742+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T11:00:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.788+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.814 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.808+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.823 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.818+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.854 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.848+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T11:00:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.898+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.916+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:21.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.926+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:21.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.958+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T11:00:22.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:21.998+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.021 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.015+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.024+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.065+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T11:00:22.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.108+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.127+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.138+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.170+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T11:00:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.227+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.273 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.268+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.282 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.277+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.317+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T11:00:22.386 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.380+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.410+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.430 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.425+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.463+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T11:00:22.536 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.530+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.557 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.552+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.562+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.610 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.604+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T11:00:22.682 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.677+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.706 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.700+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.719 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.712+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.762 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.757+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T11:00:22.858 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.853+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.901+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:22.916 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.911+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:22.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:22.964+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T11:00:23.023 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.017+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.038+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:23.054 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.048+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.088+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T11:00:23.165 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.160+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.196 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.191+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:23.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.209+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.264 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.259+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T11:00:23.341 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.330+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.362+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:23.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.385+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.445+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T11:00:23.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.520+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.550 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.544+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:23.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.556+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.602+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T11:00:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.672+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.698 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.693+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:23.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.703+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.757+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T11:00:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.843+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.901+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:23.929 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:23.923+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.024+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T11:00:24.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.087+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.109+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.123+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.167+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T11:00:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.233+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.259 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.254+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.267+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.313+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T11:00:24.360 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.354+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.374+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:24.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.382+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.430 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.424+0000 7f3c68154ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T11:00:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.489+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.519 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.513+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:00:24.528 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.523+0000 7f3c68154ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:00:24.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.596+0000 7f3c68154ec0 20 remove_watcher() i=0 2019-12-09T11:00:24.604 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.596+0000 7f3c68154ec0 2 removed watcher, disabling cache 2019-12-09T11:00:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.648+0000 7f3c68154ec0 20 remove_watcher() i=1 2019-12-09T11:00:24.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.690+0000 7f3c68154ec0 20 remove_watcher() i=2 2019-12-09T11:00:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.742+0000 7f3c68154ec0 20 remove_watcher() i=3 2019-12-09T11:00:24.797 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.791+0000 7f3c68154ec0 20 remove_watcher() i=4 2019-12-09T11:00:24.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.852+0000 7f3c68154ec0 20 remove_watcher() i=5 2019-12-09T11:00:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.890+0000 7f3c68154ec0 20 remove_watcher() i=6 2019-12-09T11:00:24.935 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T11:00:24.930+0000 7f3c68154ec0 20 remove_watcher() i=7 2019-12-09T11:00:24.945 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-12-09T11:00:25.537 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :clean 2019-12-09T11:00:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :compileJava 2019-12-09T11:00:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :processResources 2019-12-09T11:00:26.134 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :classes 2019-12-09T11:00:26.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:26.334 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :compileTestJava 2019-12-09T11:00:26.334 INFO:teuthology.orchestra.run.smithi059.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T11:00:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T11:00:26.335 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T11:00:26.335 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :testClasses 2019-12-09T11:00:29.434 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:29.434 INFO:teuthology.orchestra.run.smithi059.stdout:> Task :test 2019-12-09T11:00:29.434 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:29.434 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-12-09T11:00:34.034 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:34.034 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-12-09T11:00:45.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:45.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-12-09T11:00:49.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:49.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-12-09T11:00:57.536 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:00:57.536 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-12-09T11:01:00.834 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:01:00.834 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-12-09T11:01:11.434 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:01:11.434 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-12-09T11:01:54.121 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:18:50.738 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.133 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:18:50.750 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.138 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:18:50.756 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.139 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:18:50.737 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.147 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:18:50.765 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.161 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:18:50.759 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.256 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:18:50.873 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.292 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:18:50.890 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:54.310 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:18:50.928 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:02:17.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:17.934 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-12-09T11:02:23.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:23.934 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-12-09T11:02:27.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:27.734 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-12-09T11:02:34.434 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:34.434 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-12-09T11:02:41.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:41.434 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-12-09T11:02:50.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:50.834 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-12-09T11:02:50.934 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-12-09T11:02:52.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:52.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-12-09T11:02:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:02:55.934 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-12-09T11:03:02.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:03:02.834 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-12-09T11:03:45.634 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:03:45.634 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-12-09T11:03:53.234 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:03:53.234 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-12-09T11:03:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:03:56.934 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-12-09T11:04:07.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:07.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-12-09T11:04:10.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:10.734 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-12-09T11:04:19.734 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:19.734 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-12-09T11:04:24.234 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:24.234 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-12-09T11:04:31.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:31.634 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-12-09T11:04:34.034 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:34.034 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-12-09T11:04:42.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:42.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-12-09T11:04:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:45.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-12-09T11:04:52.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:52.234 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-12-09T11:04:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:04:55.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-12-09T11:05:02.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:05:02.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-12-09T11:05:10.634 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:05:10.634 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-12-09T11:05:15.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:05:15.139 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-12-09T11:06:00.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:06:00.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-12-09T11:06:12.434 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:06:12.434 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-12-09T11:07:06.640 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:24:03.258 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.656 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:24:03.274 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.657 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:24:03.275 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.659 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:24:03.277 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.671 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:24:03.269 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.679 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:24:03.278 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.934 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:24:03.551 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.939 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:24:03.557 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:06.978 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:24:03.576 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.659 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:24:27.277 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.673 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:24:27.281 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.743 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:24:27.361 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.750 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:24:27.348 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.755 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:24:27.354 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.777 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:24:27.394 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.783 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:24:27.400 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.807 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:24:27.405 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:07:30.818 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:24:27.435 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:45.037 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:09:45.038 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-12-09T11:10:00.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:10:00.233 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-12-09T11:10:01.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:10:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-12-09T11:10:09.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:10:09.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-12-09T11:10:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:10:15.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-12-09T11:10:28.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:10:28.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-12-09T11:11:11.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:11:11.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-12-09T11:11:21.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:11:21.533 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-12-09T11:11:21.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:11:21.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-12-09T11:11:23.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:11:23.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-12-09T11:12:13.882 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T11:12:13.567+0000 99f2b700 -1 handle_data failed with exit code -104 2019-12-09T11:12:14.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:12:14.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-12-09T11:12:19.697 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:29:16.315 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.706 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:29:16.324 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.715 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:29:16.333 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.720 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:29:16.338 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.731 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:29:16.328 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.737 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:29:16.335 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.890 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:29:16.509 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.914 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:29:16.512 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:19.917 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:29:16.536 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:12:22.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:12:22.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T11:14:17.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:14:17.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T11:14:30.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:14:30.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-12-09T11:15:33.737 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:15:33.737 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-12-09T11:15:42.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:15:42.233 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-12-09T11:15:52.353 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:32:48.971 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.362 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:32:48.981 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.405 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:32:49.023 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.412 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:32:49.031 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.455 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:32:49.053 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.457 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:32:49.055 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.522 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:32:49.140 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.560 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:32:49.178 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:15:52.636 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:32:49.234 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.889 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:34:27.507 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.901 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:34:27.519 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.906 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:34:27.524 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.909 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:34:27.527 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.922 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:34:27.519 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.945 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:34:27.543 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:30.976 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:34:27.594 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:31.003 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:34:27.621 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:17:31.075 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:34:27.673 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:21:29.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:21:29.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-12-09T11:21:38.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:21:38.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-12-09T11:22:43.663 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:39:40.282 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.672 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:39:40.290 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.680 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:39:40.298 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.682 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:39:40.301 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.683 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:39:40.280 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.704 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:39:40.301 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.892 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:39:40.489 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.935 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:39:40.554 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:22:43.947 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:39:40.565 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:23:15.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:23:15.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-12-09T11:23:23.832 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:23:23.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-12-09T11:24:13.941 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:41:10.560 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:13.981 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:41:10.599 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.036 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:41:10.633 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.038 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:41:10.656 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.043 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:41:10.640 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.051 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:41:10.670 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.054 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:41:10.673 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.111 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:41:10.708 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:14.120 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:41:10.739 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:45.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:24:45.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-12-09T11:24:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:24:55.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-12-09T11:25:04.324 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:42:00.943 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.371 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:42:00.990 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.384 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:42:00.981 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.400 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:42:01.019 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.408 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:42:01.027 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.408 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:42:01.005 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.426 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:42:01.045 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.461 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:42:01.057 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:04.468 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:42:01.086 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:25:09.332 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:25:09.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-12-09T11:25:21.532 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:25:21.533 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-12-09T11:26:38.932 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:26:38.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-12-09T11:26:59.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:26:59.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-12-09T11:27:05.332 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:27:05.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-12-09T11:27:52.862 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:44:49.481 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:52.873 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:44:49.492 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:52.880 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:44:49.499 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:52.882 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:44:49.479 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:52.886 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:44:49.483 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:52.890 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:44:49.509 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:52.978 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:44:49.596 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:53.029 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:44:49.647 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:53.035 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:44:49.632 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:27:53.232 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:27:53.232 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-12-09T11:28:11.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:28:11.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-12-09T11:29:06.332 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:29:06.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-12-09T11:29:21.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:29:21.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-12-09T11:31:01.532 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:31:01.533 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-12-09T11:31:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:31:30.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-12-09T11:32:35.900 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:49:32.519 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:35.907 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:49:32.526 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:35.927 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:49:32.545 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:35.931 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:49:32.550 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:35.955 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:49:32.552 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:35.958 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:49:32.556 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:36.057 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:49:32.675 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:36.062 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:49:32.681 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:36.122 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:49:32.719 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:00.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:33:00.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-12-09T11:33:05.116 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:50:01.733 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.127 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:50:01.746 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.132 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:50:01.750 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.135 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:50:01.754 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.137 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:50:01.735 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.142 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:50:01.740 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.169 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:50:01.788 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.182 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:50:01.801 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:05.204 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:50:01.801 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:33:32.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:33:32.534 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-12-09T11:35:46.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:35:46.634 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-12-09T11:36:13.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:36:13.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-12-09T11:38:17.935 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:55:14.552 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:17.940 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:55:14.558 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:17.953 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:55:14.571 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:17.953 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:55:14.550 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:17.956 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:55:14.553 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:17.958 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:55:14.576 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:18.149 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:55:14.747 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:18.181 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:55:14.799 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:18.191 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:55:14.809 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:27.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:38:27.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-12-09T11:39:00.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:39:00.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-12-09T11:40:57.484 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:57:54.102 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.508 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:57:54.126 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.510 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:57:54.128 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.513 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:57:54.130 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.573 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:57:54.168 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.573 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:57:54.170 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.617 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:01:57:54.235 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.633 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:01:57:54.250 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:40:57.670 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:01:57:54.265 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:24.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:42:24.734 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-12-09T11:43:28.022 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:00:24.640 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.031 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:00:24.649 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.042 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:00:24.660 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.047 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:00:24.665 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.053 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:00:24.651 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.062 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:00:24.660 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.147 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:00:24.765 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.156 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:00:24.774 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:28.203 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:00:24.801 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:40.936 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T11:44:40.936 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-12-09T11:48:40.028 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:05:36.646 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.031 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:05:36.650 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.052 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:05:36.649 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.074 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:05:36.671 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.115 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:05:36.733 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.124 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:05:36.742 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.279 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:05:36.897 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.305 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:05:36.902 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:48:40.371 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:05:36.982 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.039 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:06:20.657 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.049 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:06:20.667 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.089 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:06:20.707 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.121 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:06:20.739 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.129 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:06:20.747 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.149 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:06:20.746 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.151 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:06:20.749 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.168 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:06:20.786 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:24.273 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:06:20.871 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.712 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:10:50.329 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.719 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:10:50.337 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.733 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:10:50.351 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.736 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:10:50.333 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.738 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:10:50.356 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.746 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:10:50.344 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.932 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:10:50.546 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.936 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:10:50.551 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:53.962 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:10:50.547 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.583 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:14:47.201 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.606 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:14:47.224 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.648 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:14:47.266 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.649 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:14:47.268 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.664 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:14:47.261 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.666 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:14:47.263 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.744 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:14:47.362 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.760 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:14:47.378 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:57:50.801 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:14:47.398 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.850 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:16:02.467 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.851 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:16:02.470 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.881 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:16:02.471 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.883 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:16:02.480 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.902 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:16:02.519 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.907 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:16:02.525 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.917 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:16:02.535 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.982 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:16:02.597 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:05.992 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:16:02.589 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:52.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:00:52.734 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-12-09T12:02:22.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:02:22.834 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-12-09T12:04:14.084 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:21:10.702 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.094 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:21:10.712 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.107 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:21:10.725 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.111 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:21:10.708 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.118 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:21:10.713 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.118 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:21:10.732 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.337 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:21:10.955 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.347 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:21:10.944 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:04:14.353 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:21:10.971 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:00.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:05:00.441 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-12-09T12:05:42.597 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:22:39.214 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.613 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:22:39.231 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.673 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:22:39.289 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.673 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:22:39.291 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.690 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:22:39.287 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.697 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:22:39.294 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.702 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:22:39.320 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.750 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:22:39.368 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:42.829 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:22:39.426 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:54.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:05:54.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-12-09T12:06:12.764 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:23:09.382 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.794 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:23:09.412 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.834 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:23:09.430 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.834 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:23:09.451 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.837 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:23:09.453 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.841 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:23:09.438 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.841 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:23:09.459 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.873 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:23:09.471 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:06:12.874 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:23:09.492 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:07:13.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:07:13.533 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-12-09T12:07:34.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:07:34.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-12-09T12:07:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:07:48.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-12-09T12:07:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:07:55.535 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-12-09T12:08:08.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:08:08.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-12-09T12:08:14.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:08:14.834 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-12-09T12:08:28.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:08:28.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-12-09T12:08:36.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:08:36.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-12-09T12:08:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:08:47.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-12-09T12:08:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:08:55.533 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-12-09T12:09:09.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:09:09.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-12-09T12:09:18.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:09:18.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-12-09T12:09:22.221 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:26:18.838 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.236 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:26:18.854 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.243 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:26:18.861 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.244 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:26:18.841 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.244 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:26:18.863 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.249 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:26:18.847 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.375 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:26:18.993 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.377 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:26:18.995 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:22.410 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:26:19.007 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:09:30.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:09:30.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-12-09T12:09:39.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:09:39.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-12-09T12:09:49.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:09:49.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-12-09T12:09:58.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:09:58.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-12-09T12:10:10.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:10:10.634 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-12-09T12:10:17.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:10:17.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-12-09T12:10:28.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:10:28.233 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-12-09T12:10:35.445 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:10:35.445 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-12-09T12:10:45.134 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:10:45.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-12-09T12:10:51.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:10:51.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-12-09T12:11:06.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:11:06.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-12-09T12:11:15.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:11:15.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-12-09T12:11:30.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:11:30.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-12-09T12:11:38.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:11:38.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-12-09T12:11:50.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:11:50.134 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-12-09T12:11:58.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:11:58.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-12-09T12:12:12.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:12:12.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-12-09T12:12:22.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:12:22.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-12-09T12:12:38.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:12:38.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-12-09T12:12:45.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:12:45.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-12-09T12:12:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:12:55.234 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-12-09T12:13:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:02.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-12-09T12:13:14.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:14.534 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-12-09T12:13:19.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:19.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-12-09T12:13:32.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:32.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-12-09T12:13:39.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:39.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-12-09T12:13:50.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-12-09T12:13:54.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:13:54.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-12-09T12:14:02.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:02.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-12-09T12:14:04.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:04.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-12-09T12:14:09.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:09.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-12-09T12:14:14.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-12-09T12:14:21.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:21.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-12-09T12:14:26.736 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:26.737 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-12-09T12:14:34.532 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:31:31.149 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.532 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:31:31.150 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.540 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:31:31.158 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.552 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:31:31.164 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.570 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:31:31.168 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.575 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:31:31.173 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.714 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:31:31.312 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.715 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:31:31.332 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:34.758 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:31:31.376 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:35.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-12-09T12:14:35.818 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:31:32.416 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.819 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:31:32.417 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.832 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:31:32.430 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.991 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:31:32.609 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.992 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:31:32.610 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.993 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:31:32.611 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:35.994 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:31:32.612 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:36.009 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:31:32.627 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:36.012 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:31:32.630 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:41.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:41.833 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-12-09T12:14:50.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:50.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-12-09T12:14:50.734 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:50.734 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-12-09T12:14:51.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:51.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-12-09T12:14:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:14:56.334 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-12-09T12:15:05.933 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:05.933 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-12-09T12:15:08.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:08.633 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-12-09T12:15:18.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:18.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-12-09T12:15:20.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:20.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-12-09T12:15:24.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:24.433 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-12-09T12:15:26.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:26.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-12-09T12:15:31.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:31.534 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-12-09T12:15:36.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:36.133 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-12-09T12:15:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:15:43.733 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-12-09T12:16:22.033 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:16:22.033 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-12-09T12:16:28.333 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:16:28.333 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-12-09T12:17:08.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:17:08.533 INFO:teuthology.orchestra.run.smithi059.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-12-09T12:17:16.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:17:16.434 INFO:teuthology.orchestra.run.smithi059.stdout:---------------------------------------------------------------------- 2019-12-09T12:17:16.434 INFO:teuthology.orchestra.run.smithi059.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-12-09T12:17:16.434 INFO:teuthology.orchestra.run.smithi059.stdout:---------------------------------------------------------------------- 2019-12-09T12:17:16.434 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:17:16.434 INFO:teuthology.orchestra.run.smithi059.stdout:BUILD SUCCESSFUL in 1h 16m 51s 2019-12-09T12:17:16.434 INFO:teuthology.orchestra.run.smithi059.stdout:5 actionable tasks: 5 executed 2019-12-09T12:17:16.444 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T12:17:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.592+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.626+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:16.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.626+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.631+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:16.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.631+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.640+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:16.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.640+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.663 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.655+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.869+0000 7f25814c6ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:17:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.869+0000 7f25814c6ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:17:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.869+0000 7f25814c6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:17:16.875 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.869+0000 7f25814c6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:17:16.875 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.869+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.882+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:16.887 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.882+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.900+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:16.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.900+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.918 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.913+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.913+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.927+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:16.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.927+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.939+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:16.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.939+0000 7f25814c6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:17:16.945 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.939+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.951+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:16.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.951+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.963+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:16.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.963+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:16.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.983+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:16.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.983+0000 7f25814c6ec0 20 zonegroup default 2019-12-09T12:17:16.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.983+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:17.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.995+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:17.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.995+0000 7f25814c6ec0 10 Cannot find current period zone using local zone 2019-12-09T12:17:17.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:16.995+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:17.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.005+0000 7f25814c6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:17.010 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.005+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:17.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.021+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:17.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.021+0000 7f25814c6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:17.038 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.033+0000 7f25814c6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:17.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.033+0000 7f25814c6ec0 20 zone default found 2019-12-09T12:17:17.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.033+0000 7f25814c6ec0 20 started sync module instance, tier type = 2019-12-09T12:17:17.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.033+0000 7f25814c6ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:17:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=0 2019-12-09T12:17:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=1 2019-12-09T12:17:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=2 2019-12-09T12:17:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=3 2019-12-09T12:17:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=4 2019-12-09T12:17:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=5 2019-12-09T12:17:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.413+0000 7f25814c6ec0 20 add_watcher() i=6 2019-12-09T12:17:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.453+0000 7f25814c6ec0 20 add_watcher() i=7 2019-12-09T12:17:17.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.453+0000 7f25814c6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:17:17.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.453+0000 7f253efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:17:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.461+0000 7f2567fff700 0 -- 172.21.15.59:0/200532670 >> v2:172.21.15.59:3300/0 conn(0x5607b38009c0 msgr2=0x5607b3800dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:17:17.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:17.509+0000 7f256e6f0700 0 -- 172.21.15.59:0/1987565644 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7f24b0022490 msgr2=0x7f24b0024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:17:19.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.129+0000 7f24ad7fa700 20 reqs_thread_entry: start 2019-12-09T12:17:19.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.129+0000 7f24a6ffd700 20 reqs_thread_entry: start 2019-12-09T12:17:19.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.129+0000 7f25814c6ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:17:19.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.130+0000 7f24a57fa700 20 reqs_thread_entry: start 2019-12-09T12:17:19.156 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.130+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:17:19.190 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.185+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.208+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.219+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.266+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:17:19.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.323+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:19.348 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.343+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:19.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1262', time=2019-12-09T13:12:17.949295+0000, chain.objs.size()=9 2019-12-09T12:17:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__multipart_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.1 2019-12-09T12:17:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__shadow_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.1_1 2019-12-09T12:17:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__multipart_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.2 2019-12-09T12:17:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__shadow_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.2_1 2019-12-09T12:17:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__multipart_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.3 2019-12-09T12:17:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__shadow_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.3_1 2019-12-09T12:17:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__multipart_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.4 2019-12-09T12:17:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__shadow_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.4_1 2019-12-09T12:17:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.357+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.72__multipart_key1.2~NYMheMUxrAs1Xjj3pMZ4fstYjpCQbpY.5 2019-12-09T12:17:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.585+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.675+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:17:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.715+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:19.737 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.732+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:19.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T12:17:19.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1164', time=2019-12-09T13:09:47.242345+0000, chain.objs.size()=5 2019-12-09T12:17:19.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.66__shadow_.K367vJGVDfkRvSgL4O_YAo_xv9hDV6e_1 2019-12-09T12:17:19.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.66__shadow_.K367vJGVDfkRvSgL4O_YAo_xv9hDV6e_2 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.66__shadow_.K367vJGVDfkRvSgL4O_YAo_xv9hDV6e_3 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.66__shadow_.K367vJGVDfkRvSgL4O_YAo_xv9hDV6e_4 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.66__shadow_.K367vJGVDfkRvSgL4O_YAo_xv9hDV6e_5 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1321', time=2019-12-09T13:17:39.634762+0000, chain.objs.size()=9 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.1 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.1_1 2019-12-09T12:17:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.2 2019-12-09T12:17:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.2_1 2019-12-09T12:17:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.3 2019-12-09T12:17:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.3_1 2019-12-09T12:17:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.753+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.4 2019-12-09T12:17:19.909 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.902+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.4_1 2019-12-09T12:17:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.943+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~ZMluVrDFbOD0kb6iVoXlOc9Mq8wTHBl.5 2019-12-09T12:17:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.959+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1380', time=2019-12-09T13:24:48.742677+0000, chain.objs.size()=9 2019-12-09T12:17:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.959+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__multipart_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.1 2019-12-09T12:17:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:19.959+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__shadow_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.1_1 2019-12-09T12:17:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.048+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__multipart_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.2 2019-12-09T12:17:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.048+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__shadow_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.2_1 2019-12-09T12:17:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.048+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__multipart_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.3 2019-12-09T12:17:20.054 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.048+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__shadow_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.3_1 2019-12-09T12:17:20.054 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.048+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__multipart_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.4 2019-12-09T12:17:20.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.054+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__shadow_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.4_1 2019-12-09T12:17:20.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.054+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.77__multipart_key1.2~jaHyE8Vg3rLBPs3ETlvfa3ZoFp2_C-R.5 2019-12-09T12:17:20.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.324+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:20.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.452+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:17:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.513+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:20.541 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.536+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:17:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ', time=2019-12-09T13:15:37.673310+0000, chain.objs.size()=9 2019-12-09T12:17:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__multipart_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.7 2019-12-09T12:17:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__shadow_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.7_1 2019-12-09T12:17:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__multipart_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.8 2019-12-09T12:17:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__shadow_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.8_1 2019-12-09T12:17:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__multipart_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.9 2019-12-09T12:17:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__shadow_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.9_1 2019-12-09T12:17:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__multipart_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.10 2019-12-09T12:17:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__shadow_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.10_1 2019-12-09T12:17:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.74__multipart_key1.2~WYPGJKmgwrlg8wPsHXFBjWlkvv3k6yQ.11 2019-12-09T12:17:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1335', time=2019-12-09T13:19:39.644772+0000, chain.objs.size()=9 2019-12-09T12:17:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.1 2019-12-09T12:17:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.1_1 2019-12-09T12:17:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.554+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.2 2019-12-09T12:17:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.706+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.2_1 2019-12-09T12:17:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.801+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.3 2019-12-09T12:17:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.801+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.3_1 2019-12-09T12:17:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.801+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.4 2019-12-09T12:17:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.801+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.4_1 2019-12-09T12:17:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:20.935+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~lQvdMmtFjgt3BVUWod43i4RvMh9Yi-K.5 2019-12-09T12:17:21.086 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.081+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.148+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:17:21.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.184+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:21.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.202+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:21.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:17:21.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1188', time=2019-12-09T13:08:55.449991+0000, chain.objs.size()=9 2019-12-09T12:17:21.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.1 2019-12-09T12:17:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.1_1 2019-12-09T12:17:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.2 2019-12-09T12:17:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.2_1 2019-12-09T12:17:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.3 2019-12-09T12:17:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.3_1 2019-12-09T12:17:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.4 2019-12-09T12:17:21.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.4_1 2019-12-09T12:17:21.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~alLs-tNVlPFSS6FDYxoPihxbIiVmXUs.5 2019-12-09T12:17:21.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1345', time=2019-12-09T13:21:29.311129+0000, chain.objs.size()=7 2019-12-09T12:17:21.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.1 2019-12-09T12:17:21.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.1_1 2019-12-09T12:17:21.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.219+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.1_2 2019-12-09T12:17:21.452 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.446+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.2 2019-12-09T12:17:21.452 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.446+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.2_1 2019-12-09T12:17:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.449+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.2_2 2019-12-09T12:17:21.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.519+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~QzOLsVPTo_P2ucPQbSN-obQ_fiCWJRC.3 2019-12-09T12:17:21.696 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.690+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.783+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:17:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.831+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:21.852 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.847+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:21.860 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.855+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.896+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:17:21.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.960+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:21.991 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:21.986+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.005+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1149', time=2019-12-09T13:06:07.578197+0000, chain.objs.size()=9 2019-12-09T12:17:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.1 2019-12-09T12:17:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.1_1 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.2 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.2_1 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.3 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.3_1 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.4 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.4_1 2019-12-09T12:17:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.006+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.5 2019-12-09T12:17:22.274 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.269+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:22.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.345+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:17:22.394 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.389+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:22.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.405+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:22.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.413+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:22.447 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.441+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:17:22.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.562+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.592+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:22.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:22.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1172', time=2019-12-09T13:07:53.774906+0000, chain.objs.size()=9 2019-12-09T12:17:22.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.1 2019-12-09T12:17:22.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.1_1 2019-12-09T12:17:22.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.2 2019-12-09T12:17:22.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.2_1 2019-12-09T12:17:22.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.3 2019-12-09T12:17:22.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.3_1 2019-12-09T12:17:22.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.4 2019-12-09T12:17:22.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.4_1 2019-12-09T12:17:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.610+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~7Z1aClSvSYY_9aUtRITKYlu2BuS2mAC.5 2019-12-09T12:17:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.843+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:22.952 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:22.947+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:17:23.020 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.015+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:23.054 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.048+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:23.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.067+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:23.127 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.122+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:17:23.211 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.205+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:23.273 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.267+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:23.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:17:23.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1194', time=2019-12-09T13:09:17.999907+0000, chain.objs.size()=7 2019-12-09T12:17:23.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.1 2019-12-09T12:17:23.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.1_1 2019-12-09T12:17:23.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.1_2 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.2 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.2_1 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.2_2 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~CCzFcGTAcnrCFCF3SOEKdUpwTgwfM7r.3 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1243', time=2019-12-09T13:11:14.831645+0000, chain.objs.size()=9 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__multipart_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.1 2019-12-09T12:17:23.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__shadow_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.1_1 2019-12-09T12:17:23.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__multipart_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.2 2019-12-09T12:17:23.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__shadow_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.2_1 2019-12-09T12:17:23.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.298+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__multipart_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.3 2019-12-09T12:17:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.447+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__shadow_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.3_1 2019-12-09T12:17:23.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.521+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__multipart_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.4 2019-12-09T12:17:23.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.521+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__shadow_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.4_1 2019-12-09T12:17:23.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.521+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.71__multipart_key1.2~LI0BNRRvb_PCEjBC2_dJd69hDZLf4a8.5 2019-12-09T12:17:23.753 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.748+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.832+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:17:23.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.880+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.900+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:23.929 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:23.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1365', time=2019-12-09T13:23:18.177131+0000, chain.objs.size()=9 2019-12-09T12:17:23.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__multipart_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.1 2019-12-09T12:17:23.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__shadow_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.1_1 2019-12-09T12:17:23.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__multipart_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.2 2019-12-09T12:17:23.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__shadow_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.2_1 2019-12-09T12:17:23.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__multipart_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.3 2019-12-09T12:17:23.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__shadow_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.3_1 2019-12-09T12:17:23.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__multipart_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.4 2019-12-09T12:17:23.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__shadow_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.4_1 2019-12-09T12:17:23.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:23.924+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.76__multipart_key1.2~VWnYEDq_xlDvuLZMLUOCuUck2DYrJJ3.5 2019-12-09T12:17:24.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.151+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.272+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:17:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.317+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:24.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.333+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1206', time=2019-12-09T13:09:54.586585+0000, chain.objs.size()=7 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.1 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.1_1 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.1_2 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.2 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.2_1 2019-12-09T12:17:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.2_2 2019-12-09T12:17:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.345+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~_U0f4k6F9PK1s0wwybfOAsM4JUAYOho.3 2019-12-09T12:17:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.569+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.649+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:17:24.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.683+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:24.704 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.700+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.724+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.724+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1328', time=2019-12-09T13:18:42.755116+0000, chain.objs.size()=9 2019-12-09T12:17:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.724+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.1 2019-12-09T12:17:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.724+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.1_1 2019-12-09T12:17:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.2 2019-12-09T12:17:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.2_1 2019-12-09T12:17:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.3 2019-12-09T12:17:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.3_1 2019-12-09T12:17:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.4 2019-12-09T12:17:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.4_1 2019-12-09T12:17:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:24.725+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~YqoQJ4ezZV0tu-15KPQ77fkymPrivGT.5 2019-12-09T12:17:25.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.097+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:25.201 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.197+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:17:25.239 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.234+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.251+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:25.269 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1180', time=2019-12-09T13:08:27.780316+0000, chain.objs.size()=9 2019-12-09T12:17:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.1 2019-12-09T12:17:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.1_1 2019-12-09T12:17:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.2 2019-12-09T12:17:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.2_1 2019-12-09T12:17:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.3 2019-12-09T12:17:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.3_1 2019-12-09T12:17:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.4 2019-12-09T12:17:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.4_1 2019-12-09T12:17:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.265+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~nEXy20MfgO_6nKTl6_YHGRj_ckixn8U.5 2019-12-09T12:17:25.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.457+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:25.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.560+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:17:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.600+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:25.620 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.616+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:25.652 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:25.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj', time=2019-12-09T13:02:16.927481+0000, chain.objs.size()=9 2019-12-09T12:17:25.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__multipart_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.1 2019-12-09T12:17:25.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__shadow_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.1_1 2019-12-09T12:17:25.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__multipart_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.2 2019-12-09T12:17:25.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__shadow_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.2_1 2019-12-09T12:17:25.653 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__multipart_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.3 2019-12-09T12:17:25.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__shadow_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.3_1 2019-12-09T12:17:25.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__multipart_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.4 2019-12-09T12:17:25.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__shadow_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.4_1 2019-12-09T12:17:25.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.52__multipart_key1.2~NPWbc00GTN4QCfQL6lSrBtvHzqQLdaj.5 2019-12-09T12:17:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.858+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:25.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.945+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:17:25.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:25.990+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.023 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.019+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.033+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.083+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:17:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.138+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.160 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.156+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:26.170 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.166+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.213 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.209+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:17:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.274+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.307 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.303+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:26.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:26.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.2198', time=2019-12-09T14:17:11.321387+0000, chain.objs.size()=9 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__multipart_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.1 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__shadow_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.1_1 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__multipart_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.2 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__shadow_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.2_1 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__multipart_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.3 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__shadow_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.3_1 2019-12-09T12:17:26.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__multipart_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.4 2019-12-09T12:17:26.325 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__shadow_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.4_1 2019-12-09T12:17:26.325 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.319+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.119__multipart_key1/data/file.mpg.2~g7QQDS66ObqGUqn4VPTB_RuG1Ka3ZoE.5 2019-12-09T12:17:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.547+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:26.667 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.662+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:17:26.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.709+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.738+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:26.750 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.746+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.799 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.795+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:17:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.837+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.854+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:26.869 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.865+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:26.898+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:17:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.037+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.060+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:27.077 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.073+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:27.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.116+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:17:27.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.220+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.256+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:27.320 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2019-12-09T12:17:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1151', time=2019-12-09T13:06:01.658306+0000, chain.objs.size()=9 2019-12-09T12:17:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.1 2019-12-09T12:17:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.1_1 2019-12-09T12:17:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.2 2019-12-09T12:17:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.2_1 2019-12-09T12:17:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.3 2019-12-09T12:17:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.3_1 2019-12-09T12:17:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.4 2019-12-09T12:17:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__shadow_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.4_1 2019-12-09T12:17:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.64__multipart_key1.2~vDt85ab9eZ51Cf2G_YZnrhJWhGbS_mw.5 2019-12-09T12:17:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1200', time=2019-12-09T13:09:43.218516+0000, chain.objs.size()=7 2019-12-09T12:17:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.1 2019-12-09T12:17:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.1_1 2019-12-09T12:17:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.315+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.1_2 2019-12-09T12:17:27.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.602+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.2 2019-12-09T12:17:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.620+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.2_1 2019-12-09T12:17:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.632+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__shadow_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.2_2 2019-12-09T12:17:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.632+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.67__multipart_key1.2~5StDDqib8SkoDVz_VOHMSZq38lWnSrS.3 2019-12-09T12:17:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.647+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u', time=2019-12-09T13:25:17.531510+0000, chain.objs.size()=20 2019-12-09T12:17:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.1 2019-12-09T12:17:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.647+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.2 2019-12-09T12:17:27.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.672+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.3 2019-12-09T12:17:27.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.672+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.4 2019-12-09T12:17:27.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.672+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.5 2019-12-09T12:17:27.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.672+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.6 2019-12-09T12:17:27.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:27.672+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.7 2019-12-09T12:17:28.123 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.8 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.9 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.10 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.11 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.12 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.13 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.14 2019-12-09T12:17:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.119+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.15 2019-12-09T12:17:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.226+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.16 2019-12-09T12:17:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.226+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.17 2019-12-09T12:17:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.226+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.18 2019-12-09T12:17:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.19 2019-12-09T12:17:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.78__multipart_key1.2~UveVWznmXYAcf1xCc9Z_UqbkTRQCd6u.20 2019-12-09T12:17:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.2182', time=2019-12-09T14:16:24.545916+0000, chain.objs.size()=9 2019-12-09T12:17:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__multipart_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.1 2019-12-09T12:17:28.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__shadow_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.1_1 2019-12-09T12:17:28.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__multipart_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.2 2019-12-09T12:17:28.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__shadow_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.2_1 2019-12-09T12:17:28.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.451+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__multipart_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.3 2019-12-09T12:17:28.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.469+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__shadow_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.3_1 2019-12-09T12:17:28.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.469+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__multipart_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.4 2019-12-09T12:17:28.473 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.469+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__shadow_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.4_1 2019-12-09T12:17:28.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.632+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.118__multipart_key1.2~eGWTU81uHzfy7cCjoFGVQhlNDqz9Tdm.5 2019-12-09T12:17:28.835 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.831+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:28.929 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:28.925+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:17:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.001+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.029+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.044+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:29.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.086+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:17:29.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.150+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:29.179 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.174+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:29.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.188+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.226+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:17:29.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.281+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.301+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:29.326 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T12:17:29.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1077', time=2019-12-09T13:03:47.731041+0000, chain.objs.size()=2 2019-12-09T12:17:29.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.56__shadow_.9iveQyVwFO8-3ZzKFE0RcAjVOhFU_fv_1 2019-12-09T12:17:29.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.56__shadow_.9iveQyVwFO8-3ZzKFE0RcAjVOhFU_fv_2 2019-12-09T12:17:29.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1291', time=2019-12-09T13:14:18.626901+0000, chain.objs.size()=9 2019-12-09T12:17:29.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__multipart_file.mpg.fNj2hEHOur9wvKB_1e8DlrbNcKnYylK.1 2019-12-09T12:17:29.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__shadow_file.mpg.fNj2hEHOur9wvKB_1e8DlrbNcKnYylK.1_1 2019-12-09T12:17:29.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__multipart_file.mpg.Mpq6O9w1U3VY0w1VZX7lD8hPh3WfC8S.2 2019-12-09T12:17:29.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__shadow_file.mpg.Mpq6O9w1U3VY0w1VZX7lD8hPh3WfC8S.2_1 2019-12-09T12:17:29.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__multipart_file.mpg.T2GhNoA-4ZA4Y1aShWzFOY3kdV-IqEX.3 2019-12-09T12:17:29.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__shadow_file.mpg.T2GhNoA-4ZA4Y1aShWzFOY3kdV-IqEX.3_1 2019-12-09T12:17:29.328 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__multipart_file.mpg.2~L0yY-WRJz7GJ-Zcs-lKM1XZGLtqlm_f.4 2019-12-09T12:17:29.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__shadow_file.mpg.2~L0yY-WRJz7GJ-Zcs-lKM1XZGLtqlm_f.4_1 2019-12-09T12:17:29.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.73__multipart_file.mpg.2~L0yY-WRJz7GJ-Zcs-lKM1XZGLtqlm_f.5 2019-12-09T12:17:29.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1340', time=2019-12-09T13:20:34.326870+0000, chain.objs.size()=7 2019-12-09T12:17:29.329 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.321+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.1 2019-12-09T12:17:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.606+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.1_1 2019-12-09T12:17:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.606+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.1_2 2019-12-09T12:17:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.606+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.2 2019-12-09T12:17:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.620+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.2_1 2019-12-09T12:17:29.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.620+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.2_2 2019-12-09T12:17:29.625 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.620+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~Vsz_jECVvr76xYyCU0ziZNUz5xEOUNy.3 2019-12-09T12:17:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.907+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:29.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:29.990+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:17:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.029+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.050 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.045+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:30.062 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:17:30.062 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 20 garbage collection: RGWGC::process iterating over entry tag='478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4361.1350', time=2019-12-09T13:21:32.010582+0000, chain.objs.size()=7 2019-12-09T12:17:30.062 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.1 2019-12-09T12:17:30.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.1_1 2019-12-09T12:17:30.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.1_2 2019-12-09T12:17:30.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.2 2019-12-09T12:17:30.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.2_1 2019-12-09T12:17:30.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__shadow_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.2_2 2019-12-09T12:17:30.063 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.057+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.75__multipart_key1.2~Z3xJv5xPEi8AivjJtb86MdQJaxrIcF6.3 2019-12-09T12:17:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.235+0000 7f25814c6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:17:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.337+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:17:30.390 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.386+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.415 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.411+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.422+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.453+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:17:30.499 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.495+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.517+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:30.535 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.530+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.585 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.581+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:17:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.655+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.684+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:30.708 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.702+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.739+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:17:30.799 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.795+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.825 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.821+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:30.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.837+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:30.899 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.895+0000 7f25814c6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:17:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:30.964+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:31.030 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.026+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.054+0000 7f25814c6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:31.237 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.231+0000 7f25814c6ec0 20 remove_watcher() i=0 2019-12-09T12:17:31.237 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.231+0000 7f25814c6ec0 2 removed watcher, disabling cache 2019-12-09T12:17:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.263+0000 7f25814c6ec0 20 remove_watcher() i=1 2019-12-09T12:17:31.295 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.291+0000 7f25814c6ec0 20 remove_watcher() i=2 2019-12-09T12:17:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.323+0000 7f25814c6ec0 20 remove_watcher() i=3 2019-12-09T12:17:31.358 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.354+0000 7f25814c6ec0 20 remove_watcher() i=4 2019-12-09T12:17:31.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.387+0000 7f25814c6ec0 20 remove_watcher() i=5 2019-12-09T12:17:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.427+0000 7f25814c6ec0 20 remove_watcher() i=6 2019-12-09T12:17:31.466 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.462+0000 7f25814c6ec0 20 remove_watcher() i=7 2019-12-09T12:17:31.473 INFO:teuthology.orchestra.run.smithi059:> radosgw-admin gc process --include-all 2019-12-09T12:17:31.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.574+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.603+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.603+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.611+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.611+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.622 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.618+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.618+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.633+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.826+0000 7fb969320ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:17:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.826+0000 7fb969320ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:17:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.826+0000 7fb969320ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:17:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.826+0000 7fb969320ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:17:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.826+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.836+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.836+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.850 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.846+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:31.850 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.846+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.856+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.856+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.883+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.883+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.890+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.890+0000 7fb969320ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:17:31.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.890+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.899 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.895+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.899 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.895+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.910+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:31.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.910+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.926 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.922+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.922+0000 7fb969320ec0 20 zonegroup default 2019-12-09T12:17:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.922+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.939+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.939+0000 7fb969320ec0 10 Cannot find current period zone using local zone 2019-12-09T12:17:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.939+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.945+0000 7fb969320ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:31.949 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.945+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.959 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.955+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:31.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.955+0000 7fb969320ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.964+0000 7fb969320ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.964+0000 7fb969320ec0 20 zone default found 2019-12-09T12:17:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.964+0000 7fb969320ec0 20 started sync module instance, tier type = 2019-12-09T12:17:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:31.964+0000 7fb969320ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:17:32.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=0 2019-12-09T12:17:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=1 2019-12-09T12:17:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=2 2019-12-09T12:17:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=3 2019-12-09T12:17:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=4 2019-12-09T12:17:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=5 2019-12-09T12:17:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.342+0000 7fb969320ec0 20 add_watcher() i=6 2019-12-09T12:17:32.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.384+0000 7fb969320ec0 20 add_watcher() i=7 2019-12-09T12:17:32.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.384+0000 7fb969320ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:17:32.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.384+0000 7fb926fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:17:32.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.392+0000 7fb95654a700 0 -- 172.21.15.59:0/86207636 >> v2:172.21.15.59:3300/0 conn(0x55917b5d1cf0 msgr2=0x55917b5d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:17:32.443 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:32.437+0000 7fb95654a700 0 -- 172.21.15.59:0/89808223 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7fb898022470 msgr2=0x7fb898024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:17:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:33.965+0000 7fb8957fa700 20 reqs_thread_entry: start 2019-12-09T12:17:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:33.965+0000 7fb88effd700 20 reqs_thread_entry: start 2019-12-09T12:17:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:33.965+0000 7fb969320ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:17:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:33.966+0000 7fb88d7fa700 20 reqs_thread_entry: start 2019-12-09T12:17:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:33.966+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:17:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.023+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.041+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.053+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.104 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.099+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:17:34.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.185+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.227 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.222+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:34.243 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.239+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.308 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.304+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:17:34.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.393+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.434+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:34.448 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.444+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.487+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:17:34.543 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.538+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.568 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.563+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.576+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.624+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:17:34.698 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.694+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.721+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:34.739 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.734+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.779+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:17:34.883 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.878+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.911+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:34.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.929+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:34.990+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:17:35.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.045+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.067 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.063+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:35.078 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.074+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.111 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.107+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:17:35.169 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.165+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.189+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.200+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.250+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:17:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.329+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.355 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.351+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:35.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.359+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.418+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:17:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.495+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.521+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:35.536 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.532+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.589 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.585+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:17:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.671+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.710+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.731+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.796+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:17:35.881 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.876+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.903+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:35.919+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.011+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:17:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.093+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.120+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:36.140 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.136+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.195+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:17:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.266+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.293+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:36.308 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.304+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.342+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:17:36.424 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.419+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.459 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.455+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:36.477 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.472+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.538+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:17:36.640 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.636+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.665+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:36.692 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.688+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.738 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.733+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:17:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.807+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.834+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:36.851 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.847+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.897+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:17:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.958+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:36.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.981+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:36.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:36.993+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.036 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.032+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:17:37.074 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.070+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.088+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.102 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.097+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.131 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.126+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:17:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.208+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.225+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.235+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.282 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.278+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:17:37.341 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.337+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.356 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.352+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.369 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.362+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.412+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:17:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.460+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.476+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.485+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.521+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:17:37.580 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.576+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.598+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.611 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.607+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.662 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.658+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:17:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.718+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.748 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.744+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.762+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.813+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:17:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.873+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.905+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:37.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.926+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:37.976+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:17:38.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.068+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.089+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:38.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.112+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.160+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:17:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.338+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.366 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.362+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:38.375 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.371+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.422 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.418+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:17:38.475 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.470+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.491+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:38.504 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.500+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.540+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:17:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.608+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.633 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.629+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:38.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.639+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.674+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:17:38.742 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.738+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.756+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.766+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.808 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.804+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:17:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.902+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.926+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:38.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.940+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:38.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:38.991+0000 7fb969320ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:17:39.065 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.061+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.090+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:17:39.112 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.108+0000 7fb969320ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:17:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.219+0000 7fb969320ec0 20 remove_watcher() i=0 2019-12-09T12:17:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.219+0000 7fb969320ec0 2 removed watcher, disabling cache 2019-12-09T12:17:39.282 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.277+0000 7fb969320ec0 20 remove_watcher() i=1 2019-12-09T12:17:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.319+0000 7fb969320ec0 20 remove_watcher() i=2 2019-12-09T12:17:39.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.358+0000 7fb969320ec0 20 remove_watcher() i=3 2019-12-09T12:17:39.426 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.421+0000 7fb969320ec0 20 remove_watcher() i=4 2019-12-09T12:17:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.484+0000 7fb969320ec0 20 remove_watcher() i=5 2019-12-09T12:17:39.549 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.545+0000 7fb969320ec0 20 remove_watcher() i=6 2019-12-09T12:17:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:39.614+0000 7fb969320ec0 20 remove_watcher() i=7 2019-12-09T12:17:39.628 INFO:teuthology.run_tasks:Running task s3tests... 2019-12-09T12:17:39.656 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-12-09T12:17:39.657 INFO:tasks.s3tests:Downloading s3-tests... 2019-12-09T12:17:39.658 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-12-09T12:17:39.658 INFO:teuthology.orchestra.run.smithi059:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-12-09T12:17:39.722 INFO:teuthology.orchestra.run.smithi059.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-12-09T12:17:40.400 INFO:tasks.s3tests:Creating rgw users... 2019-12-09T12:17:40.400 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-12-09T12:17:40.401 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key BSJMZOXFZWFFSFJGBTNI --secret iJkgoYT8DxZHZU34EYGPNYxv6w8Zj6dhkpqaqdfMY96vRdPC2uaX0A== --email bar.client.0+test@test.test --cluster ceph 2019-12-09T12:17:40.582 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.578+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.614+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:40.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.614+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.627+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.627+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.637+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.637+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.660+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.873+0000 7f3e8ca01ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:17:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.873+0000 7f3e8ca01ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:17:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.873+0000 7f3e8ca01ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:17:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.873+0000 7f3e8ca01ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:17:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.873+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.891+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:40.896 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.891+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.906+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.906+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.925+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.925+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.946 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.941+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:40.946 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.941+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.952 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.948+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:40.953 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.948+0000 7f3e8ca01ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:17:40.953 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.948+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.962 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.957+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:40.962 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.957+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.972 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.968+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.968+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.983+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.983+0000 7f3e8ca01ec0 20 zonegroup default 2019-12-09T12:17:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.983+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:40.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.989+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:40.993 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.989+0000 7f3e8ca01ec0 10 Cannot find current period zone using local zone 2019-12-09T12:17:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.989+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:41.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.995+0000 7f3e8ca01ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:41.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:40.995+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:41.016 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.012+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:41.016 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.012+0000 7f3e8ca01ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.022+0000 7f3e8ca01ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.022+0000 7f3e8ca01ec0 20 zone default found 2019-12-09T12:17:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.022+0000 7f3e8ca01ec0 20 started sync module instance, tier type = 2019-12-09T12:17:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.022+0000 7f3e8ca01ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:17:41.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.299+0000 7f3e8ca01ec0 20 add_watcher() i=0 2019-12-09T12:17:41.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.299+0000 7f3e8ca01ec0 20 add_watcher() i=1 2019-12-09T12:17:41.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.299+0000 7f3e8ca01ec0 20 add_watcher() i=2 2019-12-09T12:17:41.304 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.299+0000 7f3e8ca01ec0 20 add_watcher() i=3 2019-12-09T12:17:41.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.299+0000 7f3e8ca01ec0 20 add_watcher() i=4 2019-12-09T12:17:41.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.299+0000 7f3e8ca01ec0 20 add_watcher() i=5 2019-12-09T12:17:41.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.301+0000 7f3e8ca01ec0 20 add_watcher() i=6 2019-12-09T12:17:41.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.335+0000 7f3e8ca01ec0 20 add_watcher() i=7 2019-12-09T12:17:41.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.335+0000 7f3e8ca01ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:17:41.339 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.335+0000 7f3e4a7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:17:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.337+0000 7f3e7a42c700 0 -- 172.21.15.59:0/2070212545 >> v2:172.21.15.59:3300/0 conn(0x55a57d7b7e00 msgr2=0x55a57d7b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:17:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:41.375+0000 7f3e7a42c700 0 -- 172.21.15.59:0/3707191135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57d7be3f0 msgr2=0x55a57d7be800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:17:43.006 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.002+0000 7f3dc0ff9700 20 reqs_thread_entry: start 2019-12-09T12:17:43.007 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.003+0000 7f3dba7fc700 20 reqs_thread_entry: start 2019-12-09T12:17:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.003+0000 7f3e8ca01ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:17:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.003+0000 7f3db8ff9700 20 reqs_thread_entry: start 2019-12-09T12:17:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.003+0000 7f3e8ca01ec0 20 get_system_obj_state: rctx=0x7ffe7b6d9d98 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55a57d835d90 s->prefetch_data=0 2019-12-09T12:17:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.003+0000 7f3e8ca01ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T12:17:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.014+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T12:17:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.014+0000 7f3e8ca01ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.014+0000 7f3e8ca01ec0 20 get_system_obj_state: rctx=0x7ffe7b6d9da8 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x55a57d837b20 s->prefetch_data=0 2019-12-09T12:17:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.014+0000 7f3e8ca01ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T12:17:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.021+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T12:17:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.021+0000 7f3e8ca01ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T12:17:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.021+0000 7f3e8ca01ec0 20 get_system_obj_state: rctx=0x7ffe7b6d9da8 obj=default.rgw.meta:users.keys:BSJMZOXFZWFFSFJGBTNI state=0x55a57d837b20 s->prefetch_data=0 2019-12-09T12:17:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.021+0000 7f3e8ca01ec0 10 cache get: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI : miss 2019-12-09T12:17:43.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.029+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI info.flags=0x0 2019-12-09T12:17:43.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.029+0000 7f3e8ca01ec0 10 adding default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI to cache LRU end 2019-12-09T12:17:43.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.029+0000 7f3e8ca01ec0 20 get_system_obj_state: rctx=0x7ffe7b6d99b8 obj=default.rgw.meta:users.keys:BSJMZOXFZWFFSFJGBTNI state=0x55a57d837b20 s->prefetch_data=0 2019-12-09T12:17:43.034 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.029+0000 7f3e8ca01ec0 10 cache get: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI : type miss (requested=0x6, cached=0x0) 2019-12-09T12:17:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.034+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI info.flags=0x0 2019-12-09T12:17:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.034+0000 7f3e8ca01ec0 10 moving default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI to cache LRU end 2019-12-09T12:17:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.035+0000 7f3e8ca01ec0 20 get_system_obj_state: rctx=0x7ffe7b6d9b38 obj=default.rgw.meta:users.keys:BSJMZOXFZWFFSFJGBTNI state=0x55a57d837b20 s->prefetch_data=0 2019-12-09T12:17:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.035+0000 7f3e8ca01ec0 10 cache get: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI : type miss (requested=0x6, cached=0x0) 2019-12-09T12:17:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.041+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI info.flags=0x0 2019-12-09T12:17:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.041+0000 7f3e8ca01ec0 10 moving default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI to cache LRU end 2019-12-09T12:17:43.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.109+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T12:17:43.113 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.109+0000 7f3e8ca01ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:43.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.109+0000 7f3e8ca01ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-12-09T12:17:43.130 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.125+0000 7f3e5affd700 10 RGWWatcher::handle_notify() notify_id 584115552325 cookie 94169263146656 notifier 4857 bl.length()=617 2019-12-09T12:17:43.130 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.126+0000 7f3e5affd700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T12:17:43.130 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.126+0000 7f3e5affd700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:43.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.664+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T12:17:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.664+0000 7f3e8ca01ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T12:17:43.672 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.664+0000 7f3e8ca01ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T12:17:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.681+0000 7f3e5affd700 10 RGWWatcher::handle_notify() notify_id 584115552326 cookie 94169263146656 notifier 4857 bl.length()=194 2019-12-09T12:17:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.681+0000 7f3e5affd700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T12:17:43.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:43.681+0000 7f3e5affd700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T12:17:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.189+0000 7f3e8ca01ec0 10 cache put: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI info.flags=0x7 2019-12-09T12:17:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.189+0000 7f3e8ca01ec0 10 moving default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI to cache LRU end 2019-12-09T12:17:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.189+0000 7f3e8ca01ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=186 2019-12-09T12:17:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.207+0000 7f3e5affd700 10 RGWWatcher::handle_notify() notify_id 584115552332 cookie 94169263149600 notifier 4857 bl.length()=186 2019-12-09T12:17:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.207+0000 7f3e5affd700 10 cache put: name=default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI info.flags=0x7 2019-12-09T12:17:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.207+0000 7f3e5affd700 10 moving default.rgw.meta+users.keys+BSJMZOXFZWFFSFJGBTNI to cache LRU end 2019-12-09T12:17:44.357 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T12:17:44.357 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "bar.client.0", 2019-12-09T12:17:44.358 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. bar.client.0", 2019-12-09T12:17:44.358 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T12:17:44.358 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T12:17:44.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T12:17:44.358 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [], 2019-12-09T12:17:44.358 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [ 2019-12-09T12:17:44.359 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T12:17:44.359 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "bar.client.0", 2019-12-09T12:17:44.359 INFO:teuthology.orchestra.run.smithi059.stdout: "access_key": "BSJMZOXFZWFFSFJGBTNI", 2019-12-09T12:17:44.359 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "iJkgoYT8DxZHZU34EYGPNYxv6w8Zj6dhkpqaqdfMY96vRdPC2uaX0A==" 2019-12-09T12:17:44.359 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T12:17:44.359 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T12:17:44.360 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [], 2019-12-09T12:17:44.360 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T12:17:44.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T12:17:44.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T12:17:44.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T12:17:44.360 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T12:17:44.361 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T12:17:44.361 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T12:17:44.361 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T12:17:44.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T12:17:44.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T12:17:44.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T12:17:44.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T12:17:44.362 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T12:17:44.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T12:17:44.362 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T12:17:44.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T12:17:44.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T12:17:44.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T12:17:44.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T12:17:44.363 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T12:17:44.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T12:17:44.363 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T12:17:44.363 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T12:17:44.364 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:17:44.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.412+0000 7f3e8ca01ec0 20 remove_watcher() i=0 2019-12-09T12:17:44.417 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.412+0000 7f3e8ca01ec0 2 removed watcher, disabling cache 2019-12-09T12:17:44.506 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.501+0000 7f3e8ca01ec0 20 remove_watcher() i=1 2019-12-09T12:17:44.547 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.542+0000 7f3e8ca01ec0 20 remove_watcher() i=2 2019-12-09T12:17:44.602 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.598+0000 7f3e8ca01ec0 20 remove_watcher() i=3 2019-12-09T12:17:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.640+0000 7f3e8ca01ec0 20 remove_watcher() i=4 2019-12-09T12:17:44.704 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.700+0000 7f3e8ca01ec0 20 remove_watcher() i=5 2019-12-09T12:17:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.743+0000 7f3e8ca01ec0 20 remove_watcher() i=6 2019-12-09T12:17:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.775+0000 7f3e8ca01ec0 20 remove_watcher() i=7 2019-12-09T12:17:44.786 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 8758295362 --totp-seed VBYUCZ3WWDH6PPHFKFAPO5ITBY64I26HEE77UPKJI74VK6ETHPPA3XK46X6LJAU3 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T12:17:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.919+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:44.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.960+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:44.964 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.960+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:44.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.969+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:44.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.969+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.978+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.978+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:44.998+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:45.203 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.199+0000 7f8ca576dec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:17:45.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.199+0000 7f8ca576dec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:17:45.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.199+0000 7f8ca576dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:17:45.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.199+0000 7f8ca576dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:17:45.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.199+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.218 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.214+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:45.218 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.214+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.228+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:45.233 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.228+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.246 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.242+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:45.246 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.242+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.258+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:45.263 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.258+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.264+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.264+0000 7f8ca576dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:17:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.264+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.275+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.275+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.302+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.302+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.318+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:45.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.318+0000 7f8ca576dec0 20 zonegroup default 2019-12-09T12:17:45.323 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.318+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.335 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.331+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:45.335 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.331+0000 7f8ca576dec0 10 Cannot find current period zone using local zone 2019-12-09T12:17:45.336 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.331+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.344 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.340+0000 7f8ca576dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:45.345 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.340+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.358+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.358+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.375+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.375+0000 7f8ca576dec0 20 zone default found 2019-12-09T12:17:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.375+0000 7f8ca576dec0 20 started sync module instance, tier type = 2019-12-09T12:17:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.375+0000 7f8ca576dec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:17:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=0 2019-12-09T12:17:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=1 2019-12-09T12:17:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=2 2019-12-09T12:17:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=3 2019-12-09T12:17:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=4 2019-12-09T12:17:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=5 2019-12-09T12:17:45.790 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.784+0000 7f8ca576dec0 20 add_watcher() i=6 2019-12-09T12:17:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.815+0000 7f8ca576dec0 20 add_watcher() i=7 2019-12-09T12:17:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.815+0000 7f8ca576dec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:17:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.815+0000 7f8c637de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:17:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.818+0000 7f8c92196700 0 -- 172.21.15.59:0/4174877618 >> v2:172.21.15.59:3300/0 conn(0x564a053d22c0 msgr2=0x564a053d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:17:45.860 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:45.856+0000 7f8c93198700 0 -- 172.21.15.59:0/3172812193 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564a053d13a0 msgr2=0x564a053d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:17:47.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.458+0000 7f8bd1ffb700 20 reqs_thread_entry: start 2019-12-09T12:17:47.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.459+0000 7f8bcbfff700 20 reqs_thread_entry: start 2019-12-09T12:17:47.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.459+0000 7f8ca576dec0 20 init_complete bucket index max shards: 0 2019-12-09T12:17:47.464 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.459+0000 7f8bca7fc700 20 reqs_thread_entry: start 2019-12-09T12:17:47.464 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.459+0000 7f8ca576dec0 20 get_system_obj_state: rctx=0x7fff0d10fdf8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x564a05449fe0 s->prefetch_data=0 2019-12-09T12:17:47.464 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.459+0000 7f8ca576dec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T12:17:47.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.485+0000 7f8ca576dec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T12:17:47.489 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.485+0000 7f8ca576dec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:47.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.485+0000 7f8ca576dec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T12:17:47.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.485+0000 7f8ca576dec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T12:17:47.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.485+0000 7f8ca576dec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T12:17:47.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.485+0000 7f8ca576dec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:47.513 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.509+0000 7f8ca576dec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T12:17:47.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.509+0000 7f8ca576dec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T12:17:47.514 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:47.509+0000 7f8ca576dec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:49.574+0000 7f8ca576dec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T12:17:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:49.574+0000 7f8ca576dec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:49.574+0000 7f8ca576dec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-12-09T12:17:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:49.582+0000 7f8c73fff700 10 RGWWatcher::handle_notify() notify_id 592705486919 cookie 94875915368000 notifier 4640 bl.length()=631 2019-12-09T12:17:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:49.582+0000 7f8c73fff700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T12:17:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:49.582+0000 7f8c73fff700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T12:17:50.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:50.012+0000 7f8ca576dec0 20 remove_watcher() i=0 2019-12-09T12:17:50.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:50.012+0000 7f8ca576dec0 2 removed watcher, disabling cache 2019-12-09T12:17:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:50.644+0000 7f8ca576dec0 20 remove_watcher() i=1 2019-12-09T12:17:50.746 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:50.742+0000 7f8ca576dec0 20 remove_watcher() i=2 2019-12-09T12:17:50.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:50.982+0000 7f8ca576dec0 20 remove_watcher() i=3 2019-12-09T12:17:51.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.124+0000 7f8ca576dec0 20 remove_watcher() i=4 2019-12-09T12:17:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.565+0000 7f8ca576dec0 20 remove_watcher() i=5 2019-12-09T12:17:51.639 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.634+0000 7f8ca576dec0 20 remove_watcher() i=6 2019-12-09T12:17:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.687+0000 7f8ca576dec0 20 remove_watcher() i=7 2019-12-09T12:17:51.702 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-12-09T12:17:51.702 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key SWOKOQZFVMPIFBJDJCXH --secret mmHD1nljgZBaSCD25JdcHa2fg45Kdf1tzmssZhByeDcsN7lNn5UhMg== --email foo.client.0+test@test.test --cluster ceph 2019-12-09T12:17:51.843 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.837+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:51.885 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.880+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:51.886 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.880+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:51.897 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.892+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:51.898 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.892+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.904+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.904+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:51.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:51.927+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.171+0000 7f6519326ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:17:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.171+0000 7f6519326ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:17:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.171+0000 7f6519326ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:17:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.171+0000 7f6519326ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:17:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.171+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.184+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:52.189 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.184+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.199+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:52.205 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.199+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.219+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.219+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.236 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.231+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:52.237 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.231+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.248 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.243+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:52.249 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.243+0000 7f6519326ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:17:52.249 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.243+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.252+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:52.257 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.252+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.272+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:52.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.272+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.281+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.281+0000 7f6519326ec0 20 zonegroup default 2019-12-09T12:17:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.281+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.296 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.291+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:52.296 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.291+0000 7f6519326ec0 10 Cannot find current period zone using local zone 2019-12-09T12:17:52.297 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.291+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.302+0000 7f6519326ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.302+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.324 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.317+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:52.325 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.317+0000 7f6519326ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.331+0000 7f6519326ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.331+0000 7f6519326ec0 20 zone default found 2019-12-09T12:17:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.331+0000 7f6519326ec0 20 started sync module instance, tier type = 2019-12-09T12:17:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.331+0000 7f6519326ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:17:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=0 2019-12-09T12:17:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=1 2019-12-09T12:17:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=2 2019-12-09T12:17:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=3 2019-12-09T12:17:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=4 2019-12-09T12:17:52.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=5 2019-12-09T12:17:52.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.681+0000 7f6519326ec0 20 add_watcher() i=6 2019-12-09T12:17:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.715+0000 7f6519326ec0 20 add_watcher() i=7 2019-12-09T12:17:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.715+0000 7f6519326ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:17:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.715+0000 7f64d6fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:17:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.720+0000 7f6506550700 0 -- 172.21.15.59:0/108431790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55abc88d3bf0 msgr2=0x55abc88da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:17:52.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.923+0000 7f6506550700 0 --2- 172.21.15.59:0/108431790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55abc88d3bf0 0x55abc88da490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T12:17:52.974 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:52.969+0000 7f6506550700 0 -- 172.21.15.59:0/135465335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc88df320 msgr2=0x55abc88e1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:17:54.678 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.672+0000 7f64457fa700 20 reqs_thread_entry: start 2019-12-09T12:17:54.678 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.673+0000 7f643effd700 20 reqs_thread_entry: start 2019-12-09T12:17:54.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.673+0000 7f6519326ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:17:54.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.674+0000 7f643d7fa700 20 reqs_thread_entry: start 2019-12-09T12:17:54.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.674+0000 7f6519326ec0 20 get_system_obj_state: rctx=0x7ffe0f6666c8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55abc8954bb0 s->prefetch_data=0 2019-12-09T12:17:54.679 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.674+0000 7f6519326ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T12:17:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.690+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T12:17:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.690+0000 7f6519326ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.690+0000 7f6519326ec0 20 get_system_obj_state: rctx=0x7ffe0f6666d8 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x55abc8956940 s->prefetch_data=0 2019-12-09T12:17:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.690+0000 7f6519326ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T12:17:54.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.703+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T12:17:54.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.703+0000 7f6519326ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T12:17:54.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.703+0000 7f6519326ec0 20 get_system_obj_state: rctx=0x7ffe0f6666d8 obj=default.rgw.meta:users.keys:SWOKOQZFVMPIFBJDJCXH state=0x55abc8956940 s->prefetch_data=0 2019-12-09T12:17:54.713 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.703+0000 7f6519326ec0 10 cache get: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH : miss 2019-12-09T12:17:54.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.716+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH info.flags=0x0 2019-12-09T12:17:54.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.716+0000 7f6519326ec0 10 adding default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH to cache LRU end 2019-12-09T12:17:54.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.716+0000 7f6519326ec0 20 get_system_obj_state: rctx=0x7ffe0f6662e8 obj=default.rgw.meta:users.keys:SWOKOQZFVMPIFBJDJCXH state=0x55abc8956940 s->prefetch_data=0 2019-12-09T12:17:54.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.716+0000 7f6519326ec0 10 cache get: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH : type miss (requested=0x6, cached=0x0) 2019-12-09T12:17:54.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.725+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH info.flags=0x0 2019-12-09T12:17:54.730 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.725+0000 7f6519326ec0 10 moving default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH to cache LRU end 2019-12-09T12:17:54.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.725+0000 7f6519326ec0 20 get_system_obj_state: rctx=0x7ffe0f666468 obj=default.rgw.meta:users.keys:SWOKOQZFVMPIFBJDJCXH state=0x55abc8956940 s->prefetch_data=0 2019-12-09T12:17:54.731 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.725+0000 7f6519326ec0 10 cache get: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH : type miss (requested=0x6, cached=0x0) 2019-12-09T12:17:54.739 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.734+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH info.flags=0x0 2019-12-09T12:17:54.740 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.734+0000 7f6519326ec0 10 moving default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH to cache LRU end 2019-12-09T12:17:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.804+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T12:17:54.810 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.804+0000 7f6519326ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:54.810 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.804+0000 7f6519326ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-12-09T12:17:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.819+0000 7f64e77fe700 10 RGWWatcher::handle_notify() notify_id 597000454195 cookie 94196292399648 notifier 4899 bl.length()=617 2019-12-09T12:17:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.819+0000 7f64e77fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T12:17:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.819+0000 7f64e77fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:54.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.931+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T12:17:54.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.931+0000 7f6519326ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T12:17:54.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.931+0000 7f6519326ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T12:17:54.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.944+0000 7f64e77fe700 10 RGWWatcher::handle_notify() notify_id 597000454173 cookie 94196292415344 notifier 4899 bl.length()=194 2019-12-09T12:17:54.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.944+0000 7f64e77fe700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T12:17:54.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:54.944+0000 7f64e77fe700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T12:17:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.115+0000 7f6519326ec0 10 cache put: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH info.flags=0x7 2019-12-09T12:17:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.115+0000 7f6519326ec0 10 moving default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH to cache LRU end 2019-12-09T12:17:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.115+0000 7f6519326ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=186 2019-12-09T12:17:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.122+0000 7f64e77fe700 10 RGWWatcher::handle_notify() notify_id 597000454196 cookie 94196292399648 notifier 4899 bl.length()=186 2019-12-09T12:17:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.122+0000 7f64e77fe700 10 cache put: name=default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH info.flags=0x7 2019-12-09T12:17:55.128 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.122+0000 7f64e77fe700 10 moving default.rgw.meta+users.keys+SWOKOQZFVMPIFBJDJCXH to cache LRU end 2019-12-09T12:17:55.560 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T12:17:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "foo.client.0", 2019-12-09T12:17:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. foo.client.0", 2019-12-09T12:17:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [], 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [ 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "foo.client.0", 2019-12-09T12:17:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "access_key": "SWOKOQZFVMPIFBJDJCXH", 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "mmHD1nljgZBaSCD25JdcHa2fg45Kdf1tzmssZhByeDcsN7lNn5UhMg==" 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [], 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T12:17:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T12:17:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T12:17:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T12:17:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T12:17:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T12:17:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T12:17:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T12:17:55.565 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T12:17:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:17:55.612 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.607+0000 7f6519326ec0 20 remove_watcher() i=0 2019-12-09T12:17:55.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.607+0000 7f6519326ec0 2 removed watcher, disabling cache 2019-12-09T12:17:55.683 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.678+0000 7f6519326ec0 20 remove_watcher() i=1 2019-12-09T12:17:55.739 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.734+0000 7f6519326ec0 20 remove_watcher() i=2 2019-12-09T12:17:55.813 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.805+0000 7f6519326ec0 20 remove_watcher() i=3 2019-12-09T12:17:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.864+0000 7f6519326ec0 20 remove_watcher() i=4 2019-12-09T12:17:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.923+0000 7f6519326ec0 20 remove_watcher() i=5 2019-12-09T12:17:56.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:55.994+0000 7f6519326ec0 20 remove_watcher() i=6 2019-12-09T12:17:56.049 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.044+0000 7f6519326ec0 20 remove_watcher() i=7 2019-12-09T12:17:56.061 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 0203964465 --totp-seed DG7KVJOROMN5JZB4F7TYLICOIUKXSZBTGEOXTLAU6AHRQIBAFS4G33E4MLUKRE55 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T12:17:56.225 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.219+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.263+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.263+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.273+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.278 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.273+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.281+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.281+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.305 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.300+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:56.494 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.489+0000 7ff0442a9ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:17:56.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.489+0000 7ff0442a9ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:17:56.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.489+0000 7ff0442a9ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:17:56.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.489+0000 7ff0442a9ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:17:56.495 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.489+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.505+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:56.511 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.505+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.521+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.521+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.540 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.533+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:56.541 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.533+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.557 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.548+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:56.557 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.548+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.554+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.554+0000 7ff0442a9ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:17:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.554+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.562+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.567 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.562+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.578 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.573+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:56.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.573+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.590 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.585+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:17:56.591 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.585+0000 7ff0442a9ec0 20 zonegroup default 2019-12-09T12:17:56.591 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.585+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.592+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.592+0000 7ff0442a9ec0 10 Cannot find current period zone using local zone 2019-12-09T12:17:56.597 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.592+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.600+0000 7ff0442a9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:56.605 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.600+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.614 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.609+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:17:56.615 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.609+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.618+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:17:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.618+0000 7ff0442a9ec0 20 zone default found 2019-12-09T12:17:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.618+0000 7ff0442a9ec0 20 started sync module instance, tier type = 2019-12-09T12:17:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.618+0000 7ff0442a9ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:17:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=0 2019-12-09T12:17:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=1 2019-12-09T12:17:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=2 2019-12-09T12:17:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=3 2019-12-09T12:17:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=4 2019-12-09T12:17:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=5 2019-12-09T12:17:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:56.990+0000 7ff0442a9ec0 20 add_watcher() i=6 2019-12-09T12:17:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:57.023+0000 7ff0442a9ec0 20 add_watcher() i=7 2019-12-09T12:17:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:57.023+0000 7ff0442a9ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:17:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:57.023+0000 7ff001fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:17:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:57.035+0000 7ff0314d3700 0 -- 172.21.15.59:0/262538374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x556fc7317f90 msgr2=0x556fc731e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:17:57.064 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:57.059+0000 7ff031cd4700 0 -- 172.21.15.59:0/4025205204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556fc731da90 msgr2=0x556fc731dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T12:17:58.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.803+0000 7fef73fff700 20 reqs_thread_entry: start 2019-12-09T12:17:58.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.803+0000 7fef71ffb700 20 reqs_thread_entry: start 2019-12-09T12:17:58.809 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.803+0000 7ff0442a9ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:17:58.810 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.804+0000 7fef5ffff700 20 reqs_thread_entry: start 2019-12-09T12:17:58.810 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.804+0000 7ff0442a9ec0 20 get_system_obj_state: rctx=0x7ffe9ba8b228 obj=default.rgw.meta:users.uid:foo.client.0 state=0x556fc73957f0 s->prefetch_data=0 2019-12-09T12:17:58.810 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.804+0000 7ff0442a9ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T12:17:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.827+0000 7ff0442a9ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T12:17:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.827+0000 7ff0442a9ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.827+0000 7ff0442a9ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T12:17:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.827+0000 7ff0442a9ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T12:17:58.834 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.827+0000 7ff0442a9ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T12:17:58.834 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.827+0000 7ff0442a9ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.851+0000 7ff0442a9ec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T12:17:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.851+0000 7ff0442a9ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T12:17:58.857 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.851+0000 7ff0442a9ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.998+0000 7ff0442a9ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T12:17:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.998+0000 7ff0442a9ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:58.998+0000 7ff0442a9ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-12-09T12:17:59.024 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.018+0000 7ff0127fc700 10 RGWWatcher::handle_notify() notify_id 597000454197 cookie 93938571572096 notifier 4942 bl.length()=631 2019-12-09T12:17:59.024 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.019+0000 7ff0127fc700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T12:17:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.019+0000 7ff0127fc700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T12:17:59.154 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.149+0000 7ff0442a9ec0 20 remove_watcher() i=0 2019-12-09T12:17:59.155 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.149+0000 7ff0442a9ec0 2 removed watcher, disabling cache 2019-12-09T12:17:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.202+0000 7ff0442a9ec0 20 remove_watcher() i=1 2019-12-09T12:17:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.257+0000 7ff0442a9ec0 20 remove_watcher() i=2 2019-12-09T12:17:59.326 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.321+0000 7ff0442a9ec0 20 remove_watcher() i=3 2019-12-09T12:17:59.372 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.366+0000 7ff0442a9ec0 20 remove_watcher() i=4 2019-12-09T12:17:59.438 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.433+0000 7ff0442a9ec0 20 remove_watcher() i=5 2019-12-09T12:17:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.487+0000 7ff0442a9ec0 20 remove_watcher() i=6 2019-12-09T12:17:59.545 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.539+0000 7ff0442a9ec0 20 remove_watcher() i=7 2019-12-09T12:17:59.557 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-12-09T12:17:59.557 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key KDNUNQRQMUHWVVRWJHLM --secret iV+7jtJ+0nDU46iiivRD1dnZZNa9OlpqH3ngni/nY9FDoCgzs3eQSw== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-12-09T12:17:59.759 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.742+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:59.776 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.769+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:59.776 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.769+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:59.780 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.775+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:59.780 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.775+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:59.786 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.781+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:17:59.786 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.781+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:17:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:17:59.797+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.017+0000 7f568af67ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:18:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.017+0000 7f568af67ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:18:00.023 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.017+0000 7f568af67ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:18:00.023 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.017+0000 7f568af67ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:18:00.023 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.017+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.033+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:00.039 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.033+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.054+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:18:00.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.054+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.066+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:00.072 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.066+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.086+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:18:00.091 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.086+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.096+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:00.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.096+0000 7f568af67ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:18:00.101 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.096+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.105+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.105+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.132+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.132+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.143+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:18:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.143+0000 7f568af67ec0 20 zonegroup default 2019-12-09T12:18:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.143+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.151+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.151+0000 7f568af67ec0 10 Cannot find current period zone using local zone 2019-12-09T12:18:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.151+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.159+0000 7f568af67ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.159+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.170+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:00.175 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.170+0000 7f568af67ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.181+0000 7f568af67ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:18:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.182+0000 7f568af67ec0 20 zone default found 2019-12-09T12:18:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.182+0000 7f568af67ec0 20 started sync module instance, tier type = 2019-12-09T12:18:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.182+0000 7f568af67ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:18:00.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=0 2019-12-09T12:18:00.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=1 2019-12-09T12:18:00.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=2 2019-12-09T12:18:00.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=3 2019-12-09T12:18:00.527 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=4 2019-12-09T12:18:00.528 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=5 2019-12-09T12:18:00.528 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.521+0000 7f568af67ec0 20 add_watcher() i=6 2019-12-09T12:18:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.553+0000 7f568af67ec0 20 add_watcher() i=7 2019-12-09T12:18:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.553+0000 7f568af67ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:18:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.553+0000 7f5648fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:18:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.567+0000 7f566f990700 0 -- 172.21.15.59:0/1881163583 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a76ff5cfe0 msgr2=0x55a76ff5f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:18:00.789 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.782+0000 7f5678992700 0 --2- 172.21.15.59:0/1881163583 >> v2:172.21.15.161:3300/0 conn(0x55a76ff55bb0 0x55a76ff55fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T12:18:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:00.813+0000 7f566f990700 0 -- 172.21.15.59:0/1881163583 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x55a76ff68020 msgr2=0x55a76ff6a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:18:02.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.569+0000 7f55baffd700 20 reqs_thread_entry: start 2019-12-09T12:18:02.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.570+0000 7f55b8ff9700 20 reqs_thread_entry: start 2019-12-09T12:18:02.575 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.570+0000 7f568af67ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:18:02.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.570+0000 7f55aeffd700 20 reqs_thread_entry: start 2019-12-09T12:18:02.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.570+0000 7f568af67ec0 20 get_system_obj_state: rctx=0x7ffe635a7a88 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55a76ffd1770 s->prefetch_data=0 2019-12-09T12:18:02.576 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.570+0000 7f568af67ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T12:18:02.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.576+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-12-09T12:18:02.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.576+0000 7f568af67ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:02.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.576+0000 7f568af67ec0 20 get_system_obj_state: rctx=0x7ffe635a7a98 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x55a76ffd3520 s->prefetch_data=0 2019-12-09T12:18:02.581 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.576+0000 7f568af67ec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-12-09T12:18:02.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.582+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-12-09T12:18:02.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.582+0000 7f568af67ec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T12:18:02.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.582+0000 7f568af67ec0 20 get_system_obj_state: rctx=0x7ffe635a7a98 obj=default.rgw.meta:users.keys:KDNUNQRQMUHWVVRWJHLM state=0x55a76ffd3520 s->prefetch_data=0 2019-12-09T12:18:02.587 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.582+0000 7f568af67ec0 10 cache get: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM : miss 2019-12-09T12:18:02.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.587+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM info.flags=0x0 2019-12-09T12:18:02.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.587+0000 7f568af67ec0 10 adding default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM to cache LRU end 2019-12-09T12:18:02.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.587+0000 7f568af67ec0 20 get_system_obj_state: rctx=0x7ffe635a76a8 obj=default.rgw.meta:users.keys:KDNUNQRQMUHWVVRWJHLM state=0x55a76ffd3520 s->prefetch_data=0 2019-12-09T12:18:02.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.587+0000 7f568af67ec0 10 cache get: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM : type miss (requested=0x6, cached=0x0) 2019-12-09T12:18:02.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.593+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM info.flags=0x0 2019-12-09T12:18:02.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.593+0000 7f568af67ec0 10 moving default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM to cache LRU end 2019-12-09T12:18:02.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.593+0000 7f568af67ec0 20 get_system_obj_state: rctx=0x7ffe635a7828 obj=default.rgw.meta:users.keys:KDNUNQRQMUHWVVRWJHLM state=0x55a76ffd3520 s->prefetch_data=0 2019-12-09T12:18:02.598 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.593+0000 7f568af67ec0 10 cache get: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM : type miss (requested=0x6, cached=0x0) 2019-12-09T12:18:02.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.598+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM info.flags=0x0 2019-12-09T12:18:02.603 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.598+0000 7f568af67ec0 10 moving default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM to cache LRU end 2019-12-09T12:18:02.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.649+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T12:18:02.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.649+0000 7f568af67ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:02.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.649+0000 7f568af67ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-12-09T12:18:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.659+0000 7f566d98c700 10 RGWWatcher::handle_notify() notify_id 597000454216 cookie 94177626166464 notifier 4673 bl.length()=691 2019-12-09T12:18:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.659+0000 7f566d98c700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T12:18:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:02.659+0000 7f566d98c700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:03.210 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.205+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T12:18:03.210 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.205+0000 7f568af67ec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T12:18:03.210 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.205+0000 7f568af67ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-12-09T12:18:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.214+0000 7f566d98c700 10 RGWWatcher::handle_notify() notify_id 597000454198 cookie 94177626170544 notifier 4673 bl.length()=224 2019-12-09T12:18:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.214+0000 7f566d98c700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T12:18:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.214+0000 7f566d98c700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T12:18:03.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.607+0000 7f568af67ec0 10 cache put: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM info.flags=0x7 2019-12-09T12:18:03.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.607+0000 7f568af67ec0 10 moving default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM to cache LRU end 2019-12-09T12:18:03.613 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.607+0000 7f568af67ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=201 2019-12-09T12:18:03.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.623+0000 7f566d98c700 10 RGWWatcher::handle_notify() notify_id 597000454217 cookie 94177626166464 notifier 4673 bl.length()=201 2019-12-09T12:18:03.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.623+0000 7f566d98c700 10 cache put: name=default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM info.flags=0x7 2019-12-09T12:18:03.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.623+0000 7f566d98c700 10 moving default.rgw.meta+users.keys+KDNUNQRQMUHWVVRWJHLM to cache LRU end 2019-12-09T12:18:03.791 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T12:18:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "testx$tenanteduser.client.0", 2019-12-09T12:18:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-12-09T12:18:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-12-09T12:18:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T12:18:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T12:18:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [], 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [ 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "testx$tenanteduser.client.0", 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "access_key": "KDNUNQRQMUHWVVRWJHLM", 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "iV+7jtJ+0nDU46iiivRD1dnZZNa9OlpqH3ngni/nY9FDoCgzs3eQSw==" 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T12:18:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [], 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T12:18:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T12:18:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T12:18:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T12:18:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T12:18:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T12:18:03.797 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T12:18:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:03.824 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.819+0000 7f568af67ec0 20 remove_watcher() i=0 2019-12-09T12:18:03.824 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.819+0000 7f568af67ec0 2 removed watcher, disabling cache 2019-12-09T12:18:03.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.861+0000 7f568af67ec0 20 remove_watcher() i=1 2019-12-09T12:18:03.916 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.911+0000 7f568af67ec0 20 remove_watcher() i=2 2019-12-09T12:18:03.976 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:03.971+0000 7f568af67ec0 20 remove_watcher() i=3 2019-12-09T12:18:04.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.028+0000 7f568af67ec0 20 remove_watcher() i=4 2019-12-09T12:18:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.092+0000 7f568af67ec0 20 remove_watcher() i=5 2019-12-09T12:18:04.157 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.152+0000 7f568af67ec0 20 remove_watcher() i=6 2019-12-09T12:18:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.209+0000 7f568af67ec0 20 remove_watcher() i=7 2019-12-09T12:18:04.222 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 0971594641 --totp-seed 634ZPC56HLNOHDS6242DXSMC4QXPVP7PY5KWQHZHAU5HT7WDUMOTPRS55UA3VJX6 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T12:18:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.388+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.423 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.418+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.423 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.418+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.428+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.434 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.428+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.436+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.436+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.453+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.616+0000 7f8018309ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T12:18:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.616+0000 7f8018309ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T12:18:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.616+0000 7f8018309ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:18:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.616+0000 7f8018309ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:18:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.616+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.625+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.625+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.646 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.640+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:18:04.646 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.640+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.655+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.655+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.672+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:18:04.677 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.672+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.687 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.682+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.682+0000 7f8018309ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:18:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.682+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.697 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.692+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.697 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.692+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.706+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.706+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.716+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:18:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.716+0000 7f8018309ec0 20 zonegroup default 2019-12-09T12:18:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.716+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.735 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.730+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.735 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.730+0000 7f8018309ec0 10 Cannot find current period zone using local zone 2019-12-09T12:18:04.736 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.730+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.738+0000 7f8018309ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:18:04.745 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.738+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.763 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.758+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:18:04.763 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.758+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.773+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:18:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.773+0000 7f8018309ec0 20 zone default found 2019-12-09T12:18:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.773+0000 7f8018309ec0 20 started sync module instance, tier type = 2019-12-09T12:18:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:04.773+0000 7f8018309ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T12:18:05.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=0 2019-12-09T12:18:05.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=1 2019-12-09T12:18:05.222 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=2 2019-12-09T12:18:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=3 2019-12-09T12:18:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=4 2019-12-09T12:18:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=5 2019-12-09T12:18:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.217+0000 7f8018309ec0 20 add_watcher() i=6 2019-12-09T12:18:05.264 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.258+0000 7f8018309ec0 20 add_watcher() i=7 2019-12-09T12:18:05.264 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.258+0000 7f8018309ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:18:05.264 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.259+0000 7f7fd5fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:18:05.275 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.270+0000 7f8005d34700 0 -- 172.21.15.59:0/279222650 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5612135dfc20 msgr2=0x5612135e2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:18:05.388 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:05.383+0000 7f8005d34700 0 -- 172.21.15.59:0/3997213910 >> [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] conn(0x5612135faf10 msgr2=0x5612135fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:18:07.275 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.269+0000 7f7f47fff700 20 reqs_thread_entry: start 2019-12-09T12:18:07.275 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.270+0000 7f7f45ffb700 20 reqs_thread_entry: start 2019-12-09T12:18:07.276 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.270+0000 7f8018309ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:18:07.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.271+0000 7f7f33fff700 20 reqs_thread_entry: start 2019-12-09T12:18:07.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.271+0000 7f8018309ec0 20 get_system_obj_state: rctx=0x7ffd7e233548 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x561213661700 s->prefetch_data=0 2019-12-09T12:18:07.277 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.271+0000 7f8018309ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T12:18:07.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.295+0000 7f8018309ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T12:18:07.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.295+0000 7f8018309ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:07.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.295+0000 7f8018309ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T12:18:07.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.295+0000 7f8018309ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T12:18:07.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.295+0000 7f8018309ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T12:18:07.302 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.295+0000 7f8018309ec0 20 rados->read ofs=0 len=0 2019-12-09T12:18:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.321+0000 7f8018309ec0 20 rados_obj.operate() r=0 bl.length=491 2019-12-09T12:18:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.321+0000 7f8018309ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T12:18:07.327 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.321+0000 7f8018309ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.484+0000 7f8018309ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T12:18:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.484+0000 7f8018309ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.484+0000 7f8018309ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-12-09T12:18:07.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.502+0000 7f7fe67fc700 10 RGWWatcher::handle_notify() notify_id 597000454218 cookie 94635634270736 notifier 4682 bl.length()=705 2019-12-09T12:18:07.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.503+0000 7f7fe67fc700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T12:18:07.508 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.503+0000 7f7fe67fc700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T12:18:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.818+0000 7f8018309ec0 20 remove_watcher() i=0 2019-12-09T12:18:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.819+0000 7f8018309ec0 2 removed watcher, disabling cache 2019-12-09T12:18:07.898 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.893+0000 7f8018309ec0 20 remove_watcher() i=1 2019-12-09T12:18:07.971 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:07.966+0000 7f8018309ec0 20 remove_watcher() i=2 2019-12-09T12:18:08.024 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:08.019+0000 7f8018309ec0 20 remove_watcher() i=3 2019-12-09T12:18:08.087 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:08.082+0000 7f8018309ec0 20 remove_watcher() i=4 2019-12-09T12:18:08.141 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:08.135+0000 7f8018309ec0 20 remove_watcher() i=5 2019-12-09T12:18:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:08.199+0000 7f8018309ec0 20 remove_watcher() i=6 2019-12-09T12:18:08.250 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T12:18:08.245+0000 7f8018309ec0 20 remove_watcher() i=7 2019-12-09T12:18:08.258 INFO:tasks.s3tests:Configuring s3-tests... 2019-12-09T12:18:08.258 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-12-09T12:18:13.821 INFO:teuthology.orchestra.run.smithi059.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T12:18:13.985 INFO:teuthology.orchestra.run.smithi059.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T12:18:14.024 INFO:teuthology.orchestra.run.smithi059.stdout:Loading mirror speeds from cached hostfile 2019-12-09T12:18:14.035 INFO:teuthology.orchestra.run.smithi059.stdout: * base: mirror.ash.fastserv.com 2019-12-09T12:18:14.035 INFO:teuthology.orchestra.run.smithi059.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T12:18:14.036 INFO:teuthology.orchestra.run.smithi059.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T12:18:14.036 INFO:teuthology.orchestra.run.smithi059.stdout: * updates: mirror.cogentco.com 2019-12-09T12:18:14.711 INFO:teuthology.orchestra.run.smithi059.stdout:7 packages excluded due to repository priority protections 2019-12-09T12:18:15.061 INFO:teuthology.orchestra.run.smithi059.stdout:Resolving Dependencies 2019-12-09T12:18:15.061 INFO:teuthology.orchestra.run.smithi059.stdout:--> Running transaction check 2019-12-09T12:18:15.061 INFO:teuthology.orchestra.run.smithi059.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-12-09T12:18:15.461 INFO:teuthology.orchestra.run.smithi059.stdout:--> Finished Dependency Resolution 2019-12-09T12:18:15.697 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:15.698 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies Resolved 2019-12-09T12:18:15.698 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:15.698 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2019-12-09T12:18:15.699 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2019-12-09T12:18:15.700 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2019-12-09T12:18:15.700 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:15.700 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 23 k 2019-12-09T12:18:15.700 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 27 k 2019-12-09T12:18:15.700 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading packages: 2019-12-09T12:18:15.825 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2019-12-09T12:18:15.829 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2019-12-09T12:18:15.830 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded 2019-12-09T12:18:15.831 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2019-12-09T12:18:16.379 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T12:18:16.895 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T12:18:16.895 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:16.895 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2019-12-09T12:18:16.895 INFO:teuthology.orchestra.run.smithi059.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-12-09T12:18:16.895 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:16.895 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2019-12-09T12:18:17.093 INFO:teuthology.orchestra.run.smithi059.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-12-09T12:18:18.618 INFO:teuthology.orchestra.run.smithi059.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T12:18:18.906 INFO:teuthology.orchestra.run.smithi059.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T12:18:19.064 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting pip 2019-12-09T12:18:19.153 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T12:18:19.226 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: pip 2019-12-09T12:18:19.226 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: pip 9.0.1 2019-12-09T12:18:19.325 INFO:teuthology.orchestra.run.smithi059.stdout: Uninstalling pip-9.0.1: 2019-12-09T12:18:19.427 INFO:teuthology.orchestra.run.smithi059.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T12:18:20.144 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed pip-19.3.1 2019-12-09T12:18:20.427 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T12:18:20.938 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting setuptools==32.3.1 2019-12-09T12:18:20.941 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T12:18:20.991 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: setuptools 2019-12-09T12:18:20.992 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T12:18:21.007 INFO:teuthology.orchestra.run.smithi059.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T12:18:21.021 INFO:teuthology.orchestra.run.smithi059.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T12:18:21.147 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed setuptools-32.3.1 2019-12-09T12:18:21.597 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T12:18:21.780 INFO:teuthology.orchestra.run.smithi059.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T12:18:21.848 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting nose>=1.0.0 2019-12-09T12:18:21.850 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T12:18:21.947 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting boto>=2.6.0 2019-12-09T12:18:21.954 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T12:18:22.502 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting boto3>=1.0.0 2019-12-09T12:18:22.542 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/1f/a5/6b25e39aea40bc03163615c07572d90aaddc9951f5af9c21204d3da46398/boto3-1.10.34-py2.py3-none-any.whl (128kB) 2019-12-09T12:18:22.690 INFO:teuthology.orchestra.run.smithi059.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T12:18:22.997 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting gevent>=1.0 2019-12-09T12:18:23.013 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T12:18:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting isodate>=0.4.4 2019-12-09T12:18:23.310 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-12-09T12:18:23.534 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting requests>=0.14.0 2019-12-09T12:18:23.544 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-12-09T12:18:23.850 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting pytz>=2011k 2019-12-09T12:18:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-12-09T12:18:24.061 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting ordereddict 2019-12-09T12:18:24.071 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-12-09T12:18:24.382 INFO:teuthology.orchestra.run.smithi059.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-12-09T12:18:24.776 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting lxml 2019-12-09T12:18:24.791 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T12:18:25.012 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-12-09T12:18:25.024 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-12-09T12:18:25.658 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting botocore<1.14.0,>=1.13.34 2019-12-09T12:18:25.670 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/59/cb/b4772a4abc128f6fd637af9d7fc93a1db11617859af680fc2b2e6282eb95/botocore-1.13.34-py2.py3-none-any.whl (5.8MB) 2019-12-09T12:18:26.326 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-12-09T12:18:26.337 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-12-09T12:18:26.529 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T12:18:26.538 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T12:18:26.591 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting six 2019-12-09T12:18:26.629 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T12:18:26.812 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-12-09T12:18:26.823 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-12-09T12:18:26.936 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting idna<2.9,>=2.5 2019-12-09T12:18:26.947 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-12-09T12:18:27.115 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-12-09T12:18:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-12-09T12:18:27.297 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting certifi>=2017.4.17 2019-12-09T12:18:27.308 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB) 2019-12-09T12:18:27.419 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting docutils<0.16,>=0.10 2019-12-09T12:18:27.435 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-12-09T12:18:27.711 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-12-09T12:18:27.722 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-12-09T12:18:27.845 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-12-09T12:18:27.852 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-12-09T12:18:27.890 INFO:teuthology.orchestra.run.smithi059.stdout:Building wheels for collected packages: ordereddict 2019-12-09T12:18:27.890 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for ordereddict (setup.py): started 2019-12-09T12:18:28.151 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-12-09T12:18:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=2046c4c1de0dabc8e1dd77f2df64cdf994beb15f64e2d34232974f23d63d8cde 2019-12-09T12:18:28.152 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-12-09T12:18:28.156 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully built ordereddict 2019-12-09T12:18:28.301 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-12-09T12:18:30.627 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 boto3-1.10.34 botocore-1.13.34 bunch-1.0.1 certifi-2019.11.28 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.2 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-12-09T12:18:30.936 INFO:teuthology.orchestra.run.smithi059.stdout:running develop 2019-12-09T12:18:30.992 INFO:teuthology.orchestra.run.smithi059.stdout:running egg_info 2019-12-09T12:18:30.993 INFO:teuthology.orchestra.run.smithi059.stdout:creating s3tests.egg-info 2019-12-09T12:18:30.993 INFO:teuthology.orchestra.run.smithi059.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-12-09T12:18:30.993 INFO:teuthology.orchestra.run.smithi059.stdout:writing s3tests.egg-info/PKG-INFO 2019-12-09T12:18:30.993 INFO:teuthology.orchestra.run.smithi059.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-12-09T12:18:30.993 INFO:teuthology.orchestra.run.smithi059.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-12-09T12:18:30.999 INFO:teuthology.orchestra.run.smithi059.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-12-09T12:18:30.999 INFO:teuthology.orchestra.run.smithi059.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T12:18:30.999 INFO:teuthology.orchestra.run.smithi059.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T12:18:30.999 INFO:teuthology.orchestra.run.smithi059.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T12:18:30.999 INFO:teuthology.orchestra.run.smithi059.stdout:running build_ext 2019-12-09T12:18:31.000 INFO:teuthology.orchestra.run.smithi059.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-12-09T12:18:31.000 INFO:teuthology.orchestra.run.smithi059.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-12-09T12:18:31.001 INFO:teuthology.orchestra.run.smithi059.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T12:18:31.001 INFO:teuthology.orchestra.run.smithi059.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T12:18:31.001 INFO:teuthology.orchestra.run.smithi059.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T12:18:31.001 INFO:teuthology.orchestra.run.smithi059.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T12:18:31.001 INFO:teuthology.orchestra.run.smithi059.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T12:18:31.002 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.002 INFO:teuthology.orchestra.run.smithi059.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-12-09T12:18:31.002 INFO:teuthology.orchestra.run.smithi059.stdout:Processing dependencies for s3tests==0.0.1 2019-12-09T12:18:31.052 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for isodate==0.6.0 2019-12-09T12:18:31.052 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: isodate 0.6.0 2019-12-09T12:18:31.052 INFO:teuthology.orchestra.run.smithi059.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T12:18:31.052 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.052 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.053 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for gevent==1.4.0 2019-12-09T12:18:31.053 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: gevent 1.4.0 2019-12-09T12:18:31.053 INFO:teuthology.orchestra.run.smithi059.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T12:18:31.053 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.054 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.056 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for bunch==1.0.1 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: bunch 1.0.1 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for PyYAML==5.2 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: PyYAML 5.2 2019-12-09T12:18:31.057 INFO:teuthology.orchestra.run.smithi059.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T12:18:31.058 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.058 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.059 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for boto3==1.10.34 2019-12-09T12:18:31.059 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: boto3 1.10.34 2019-12-09T12:18:31.059 INFO:teuthology.orchestra.run.smithi059.stdout:Adding boto3 1.10.34 to easy-install.pth file 2019-12-09T12:18:31.059 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.059 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.060 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for boto==2.49.0 2019-12-09T12:18:31.060 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: boto 2.49.0 2019-12-09T12:18:31.060 INFO:teuthology.orchestra.run.smithi059.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T12:18:31.062 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.062 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.062 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for six==1.13.0 2019-12-09T12:18:31.062 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: six 1.13.0 2019-12-09T12:18:31.062 INFO:teuthology.orchestra.run.smithi059.stdout:Adding six 1.13.0 to easy-install.pth file 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for greenlet==0.4.15 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: greenlet 0.4.15 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.063 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for s3transfer==0.2.1 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: s3transfer 0.2.1 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for botocore==1.13.34 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: botocore 1.13.34 2019-12-09T12:18:31.064 INFO:teuthology.orchestra.run.smithi059.stdout:Adding botocore 1.13.34 to easy-install.pth file 2019-12-09T12:18:31.065 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.065 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.065 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for jmespath==0.9.4 2019-12-09T12:18:31.065 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: jmespath 0.9.4 2019-12-09T12:18:31.066 INFO:teuthology.orchestra.run.smithi059.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-12-09T12:18:31.066 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.066 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for futures==3.3.0 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: futures 3.3.0 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for urllib3==1.25.7 2019-12-09T12:18:31.067 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: urllib3 1.25.7 2019-12-09T12:18:31.068 INFO:teuthology.orchestra.run.smithi059.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-12-09T12:18:31.069 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.069 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.069 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for python-dateutil==2.8.0 2019-12-09T12:18:31.069 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: python-dateutil 2.8.0 2019-12-09T12:18:31.069 INFO:teuthology.orchestra.run.smithi059.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout:Searching for docutils==0.15.2 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout:Best match: docutils 0.15.2 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T12:18:31.070 INFO:teuthology.orchestra.run.smithi059.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T12:18:31.071 INFO:teuthology.orchestra.run.smithi059.stdout:Finished processing dependencies for s3tests==0.0.1 2019-12-09T12:18:31.077 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-12-09T12:18:31.152 INFO:tasks.s3tests:Configuring boto... 2019-12-09T12:18:31.159 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T12:18:31.245 INFO:teuthology.orchestra.run.smithi059:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-12-09T12:18:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-12-09T12:18:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-12-09T12:18:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-12-09T12:18:46.783 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-12-09T12:18:50.105 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-12-09T12:18:52.243 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-12-09T12:18:58.386 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-12-09T12:19:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-12-09T12:19:00.719 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-12-09T12:19:01.026 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-12-09T12:19:04.400 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-12-09T12:19:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-12-09T12:19:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T12:19:12.864 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-12-09T12:19:13.577 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T12:19:13.640 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-12-09T12:19:13.640 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-12-09T12:19:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-12-09T12:19:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-12-09T12:19:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T12:19:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T12:19:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-12-09T12:19:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T12:19:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T12:19:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-12-09T12:19:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T12:19:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-12-09T12:19:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T12:19:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T12:19:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T12:19:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T12:19:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T12:19:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T12:19:13.645 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T12:19:13.647 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-12-09T12:19:13.647 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-12-09T12:19:13.647 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-12-09T12:19:13.647 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-12-09T12:19:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T12:19:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T12:19:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-12-09T12:19:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T12:19:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T12:19:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-12-09T12:19:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T12:19:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-12-09T12:19:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T12:19:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T12:19:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T12:19:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T12:19:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T12:19:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T12:19:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T12:19:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-12-09T12:19:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-12-09T12:19:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-12-09T12:19:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-12-09T12:19:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-12-09T12:19:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-12-09T12:19:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-12-09T12:19:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-12-09T12:19:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-12-09T12:19:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-12-09T12:19:37.495 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-12-09T12:19:40.874 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:36:37.491 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:40.884 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:36:37.502 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:40.891 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:36:37.509 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:40.894 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:36:37.512 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:40.896 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:36:37.494 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:40.913 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:36:37.511 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:41.097 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:36:37.715 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:41.104 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:36:37.722 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:41.154 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:36:37.752 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:43.152 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-12-09T12:22:14.454 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-12-09T12:22:56.188 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:39:52.805 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.191 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:39:52.809 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.226 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:39:52.844 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.230 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:39:52.848 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.277 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:39:52.875 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.285 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:39:52.883 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.323 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:39:52.940 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.428 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:39:53.046 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:22:56.431 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:39:53.030 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.735 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:41:49.352 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.744 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:41:49.362 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.753 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:41:49.371 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.757 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:41:49.375 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.766 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:41:49.364 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.773 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:41:49.371 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.837 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:41:49.454 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.859 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:41:49.477 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:52.910 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:41:49.508 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:41.256 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-12-09T12:25:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-12-09T12:25:54.483 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-12-09T12:25:59.799 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-12-09T12:26:03.299 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-12-09T12:26:03.299 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-12-09T12:26:03.299 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-12-09T12:26:03.299 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-12-09T12:26:03.299 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-12-09T12:26:03.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-12-09T12:26:03.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-12-09T12:26:03.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-12-09T12:26:03.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T12:26:03.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T12:26:03.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T12:26:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T12:26:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T12:26:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-12-09T12:26:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-12-09T12:26:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-12-09T12:26:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-12-09T12:26:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-12-09T12:26:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-12-09T12:26:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-12-09T12:26:03.305 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-12-09T12:26:03.305 INFO:teuthology.orchestra.run.smithi059.stderr:SKIP 2019-12-09T12:26:03.310 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-12-09T12:27:17.802 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T12:27:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T12:27:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T12:27:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T12:27:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T12:27:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T12:27:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T12:27:18.167 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T12:27:18.337 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T12:27:18.691 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T12:27:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T12:27:18.738 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T12:27:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T12:27:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T12:27:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T12:27:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T12:27:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T12:27:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T12:27:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T12:27:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T12:27:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T12:27:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T12:27:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T12:27:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T12:27:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T12:27:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T12:27:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T12:27:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T12:27:18.750 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T12:27:18.750 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T12:27:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T12:27:18.798 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T12:27:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T12:27:18.951 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T12:27:19.662 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:19.663 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:20.907 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:20.907 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:21.511 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:21.512 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:22.551 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:25.187 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:25.187 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:27:25.741 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:25.741 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:28.931 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:27:36.806 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:36.806 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:39.236 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:39.236 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:39.367 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:27:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:39.583 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:41.408 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:41.409 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:42.981 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:45.107 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:27:45.404 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:45.404 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:48.845 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:50.541 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:27:53.336 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:53.336 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:57.281 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:57.281 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:27:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:27:59.245 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:01.352 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:01.677 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:05.505 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:07.737 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:07.737 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:09.021 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:09.290 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:11.126 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:11.126 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:11.219 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:11.504 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:11.504 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:13.764 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:13.764 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:15.915 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:18.515 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:18.792 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:18.792 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:22.046 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:22.046 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:22.868 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:23.261 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:25.922 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:27.079 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:27.080 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:29.035 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:29.452 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:32.411 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:32.411 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:34.663 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:35.068 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:35.068 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:35.715 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:38.971 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:38.971 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:39.120 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:39.466 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:42.564 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:42.564 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:44.736 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:44.736 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:47.148 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:47.148 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:51.746 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:51.746 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:53.655 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:54.366 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:54.366 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:56.869 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:56.870 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:57.139 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:57.502 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:59.471 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:28:59.574 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:28:59.968 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:28:59.969 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:02.534 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:02.535 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:02.626 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:02.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:05.920 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:05.920 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:06.687 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:06.687 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:09.171 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:09.726 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:09.726 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:12.185 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:12.454 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:13.081 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:13.081 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:13.222 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:13.541 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:13.821 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:14.252 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:14.252 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:14.375 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:14.909 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:15.113 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:15.398 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:15.399 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:19.326 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:19.622 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:19.623 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:23.278 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:23.278 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:28.052 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:28.900 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:28.900 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:36.707 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:36.707 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:38.044 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:38.367 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:38.368 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:46.637 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:46.637 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:47.159 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:47.396 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:29:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:29:59.447 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:29:59.448 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:06.542 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:30:07.423 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:07.423 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:07.607 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:47:04.225 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.612 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:47:04.230 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.619 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:47:04.237 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.626 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:47:04.244 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.636 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:47:04.233 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.643 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:47:04.242 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.825 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:47:04.442 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:07.829 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:47:04.447 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:08.028 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:47:04.626 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:30:16.766 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:16.766 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:17.949 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:30:18.331 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:18.331 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:27.674 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:27.675 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:28.968 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:28.968 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:30.435 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:32.002 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:32.002 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:33.196 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:33.196 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:34.215 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:34.215 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:35.366 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:36.124 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:36.125 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:30:37.101 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:37.102 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:47.916 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:47.917 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:48.850 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:48.850 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:49.520 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:50.465 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:51.068 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:51.577 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:51.577 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:52.538 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:53.507 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:30:54.362 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:30:54.784 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:30:54.785 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:00.032 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:00.344 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:31:00.729 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:00.729 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:06.072 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:06.072 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:31:07.052 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:07.052 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:12.528 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:12.528 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:12.887 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:31:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:13.275 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:17.866 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:48:14.483 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.912 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:48:14.529 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.914 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:48:14.531 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.916 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:48:14.515 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.917 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:48:14.534 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.920 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:48:14.519 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.960 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:48:14.577 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.963 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:48:14.581 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:17.970 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:48:14.569 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:19.848 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:19.849 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:20.156 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:31:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:20.461 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:30.460 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:31.198 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:31.825 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:31.826 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:33.070 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:33.070 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:33.783 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:33.783 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:34.423 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:34.424 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:35.157 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:36.877 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:36.878 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:37.080 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:31:37.403 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:37.403 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:44.642 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:44.642 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:45.970 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:46.800 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:48.018 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:49.144 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:49.144 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:50.305 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:50.305 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:51.083 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:51.084 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:52.096 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:52.096 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:31:52.694 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:31:52.937 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:31:52.937 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:02.243 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:03.053 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:32:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:03.485 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:11.232 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:11.233 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:11.627 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:32:11.971 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:11.971 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:20.813 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:20.813 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:21.817 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:32:22.095 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:22.096 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:30.906 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:30.906 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:31.235 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:32:31.717 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:31.717 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:42.965 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:42.965 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:32:44.146 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:44.146 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:52.470 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:52.470 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:32:52.805 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:32:53.076 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:32:53.077 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:01.326 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:01.326 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:03.101 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:03.401 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:03.401 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:13.453 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:13.454 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:14.189 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:14.547 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:22.347 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:22.348 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:22.950 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:23.416 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:23.416 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:30.868 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:30.868 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:31.380 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:31.381 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:40.287 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:41.392 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:41.985 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:41.985 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:48.932 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:49.749 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:49.749 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:56.839 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:56.840 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:33:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:33:58.390 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:33:58.391 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:05.664 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:05.665 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:06.653 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:34:06.893 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:06.893 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:15.032 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:15.032 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:34:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:15.882 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:22.940 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:22.940 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:24.260 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:34:24.617 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:24.617 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:31.754 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:34:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:34:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:34:33.197 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:35:19.715 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:52:16.332 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.722 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:52:16.339 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.735 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:52:16.352 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.740 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:52:16.357 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.743 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:52:16.341 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.749 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:52:16.348 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.916 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:52:16.533 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.938 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:52:16.555 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:35:19.941 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:52:16.540 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.518 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:56:36.135 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.551 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:56:36.169 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.570 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:56:36.188 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.578 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:56:36.177 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.590 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:56:36.188 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.598 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:56:36.216 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.727 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:56:36.341 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.735 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:56:36.333 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:39:39.756 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:56:36.371 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.228 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:57:26.846 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.238 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:57:26.854 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.248 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:57:26.866 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.250 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:57:26.849 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.251 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:57:26.869 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.259 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:57:26.857 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.291 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:02:57:26.909 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.308 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:02:57:26.926 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:30.353 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:02:57:26.952 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.361 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:02:39.979 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.368 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:02:39.986 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.376 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:02:39.995 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.383 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:02:40.002 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.388 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:02:39.986 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.393 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:02:39.992 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.597 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:02:40.215 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.612 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:02:40.230 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:43.624 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:02:40.223 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.113 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:03:37.731 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.118 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:03:37.737 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.172 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:03:37.790 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.172 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:03:37.791 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.175 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:03:37.774 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.177 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:03:37.796 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.178 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:03:37.777 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.235 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:03:37.854 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:41.263 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:03:37.862 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.517 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:05:03.135 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.559 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:05:03.177 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.582 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:05:03.201 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.585 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:05:03.204 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.606 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:05:03.205 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.612 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:05:03.211 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.622 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:05:03.240 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.644 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:05:03.263 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:48:06.677 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:05:03.275 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.501 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:07:48.120 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.510 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:07:48.129 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.523 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:07:48.142 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.523 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:07:48.122 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.527 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:07:48.146 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.529 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:07:48.128 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.644 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:07:48.263 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.658 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:07:48.256 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:51.671 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:07:48.290 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.154 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:12:59.773 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.164 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:12:59.783 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.177 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:12:59.796 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.179 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:12:59.778 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.185 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:12:59.804 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.186 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:12:59.785 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.377 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:12:59.996 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.384 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:12:59.983 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:03.411 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:13:00.030 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.175 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:13:24.794 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.199 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:13:24.818 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.208 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:13:24.828 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.224 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:13:24.844 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.233 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:13:24.832 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.242 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:13:24.841 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.251 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:13:24.870 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.256 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:13:24.875 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:28.273 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:13:24.872 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:31.158 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:31.159 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:56:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:34.030 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:39.440 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:39.736 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:56:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:40.010 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:44.136 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:44.137 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:44.931 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:56:45.226 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:45.226 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:51.866 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:56:52.513 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:52.537 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:59.692 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:56:59.692 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:56:59.998 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:00.237 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:07.786 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:09.177 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:09.178 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:15.753 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:15.754 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:15.943 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:16.513 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:16.513 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:22.225 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:22.225 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:23.058 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:23.353 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:23.354 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:31.046 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:31.046 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:31.718 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:31.718 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:36.946 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:36.946 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:37.601 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:37.873 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:43.474 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:43.475 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:44.639 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:44.639 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:51.718 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:51.718 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:57:52.866 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:57:53.585 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:57:53.585 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:00.230 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:00.231 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:00.697 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:00.922 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:00.923 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:07.562 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:07.563 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:08.990 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:09.314 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:09.315 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:17.201 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:17.687 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:18.404 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:18.405 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:24.198 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:24.681 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:25.019 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:25.020 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:32.815 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:32.815 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:33.473 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:33.816 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:39.428 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:39.947 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:49.402 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:49.664 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:50.388 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:50.388 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:56.846 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:58:57.540 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:58:57.846 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:58:57.846 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:06.850 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:59:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:07.123 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:13.887 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:14.989 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:59:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:15.302 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:26.062 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:26.063 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:59:26.786 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:26.787 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:33.698 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:35.411 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:59:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:35.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:43.768 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:43.769 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:45.387 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:59:45.847 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:45.847 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:55.288 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:55.288 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T12:59:55.508 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T12:59:55.768 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T12:59:55.768 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:03.359 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:03.359 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:00:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:04.824 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:11.763 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:11.763 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:00:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:12.435 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:20.833 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:20.833 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:21.258 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:00:21.569 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:21.570 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:00:57.795 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:00:57.796 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:00.299 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:01:00.723 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:00.723 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:11.848 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:18:08.462 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:11.857 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:18:08.472 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:11.864 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:18:08.483 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:11.867 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:18:08.465 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:11.868 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:18:08.487 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:11.874 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:18:08.473 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:12.129 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:18:08.748 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:12.130 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:18:08.730 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:12.132 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:18:08.752 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:38.432 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:41.222 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:01:41.942 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:41.942 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:48.369 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:48.710 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:01:48.983 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:48.984 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:56.869 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:56.869 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:01:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:01:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:01:57.726 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:05.281 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:05.281 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:02:07.304 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:07.304 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:15.612 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:15.612 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:16.369 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:02:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:16.733 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:23.385 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:23.386 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:02:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:24.377 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:33.379 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:02:34.316 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:34.317 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:42.379 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:42.379 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:43.121 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:02:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:43.568 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:52.096 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:52.097 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:02:52.581 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:02:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:02:52.984 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:01.927 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:01.927 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:02.292 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:03.053 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:03.053 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:10.796 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:11.482 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:11.482 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:19.744 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:19.745 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:23.587 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:23.828 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:23.829 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:26.089 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:26.089 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:27.309 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:27.309 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:40.773 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:45.835 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:46.104 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:46.104 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:49.003 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:50.263 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:51.132 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:51.416 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:51.417 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:54.400 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:55.660 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:55.660 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:56.369 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:58.423 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:58.424 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:03:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:03:59.463 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:03:59.672 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:02.615 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:02.615 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:03.359 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:03.895 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:03.896 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:04.225 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:04.833 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:04.833 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:05.450 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:05.923 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:05.923 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:06.694 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:07.300 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:07.300 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:11.744 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:12.686 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:12.941 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:12.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:15.778 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:16.119 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:16.119 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:16.534 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:16.535 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:16.818 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:16.818 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:17.143 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:17.611 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:18.164 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:18.344 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:18.606 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:18.606 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:19.331 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:19.331 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:19.942 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:20.269 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:20.269 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:20.503 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:20.759 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:21.926 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:22.273 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:22.273 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:22.659 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:22.659 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:23.175 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:23.175 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:23.203 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:23.833 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:23.834 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:24.120 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:24.120 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:24.919 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:24.919 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:25.864 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:25.865 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:26.928 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:27.436 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:27.436 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:27.693 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:27.693 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:28.127 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:28.504 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:28.505 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:28.971 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:28.971 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:29.315 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:29.315 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:29.894 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:29.894 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:30.330 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:30.330 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:30.627 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:31.692 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:32.076 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:32.805 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:33.250 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:33.718 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:33.718 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:34.080 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:34.338 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:34.911 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:34.911 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:35.966 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:36.563 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:36.767 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:37.457 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:37.974 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:38.007 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:38.409 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:38.649 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:39.025 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:39.025 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:39.404 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:39.405 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:43.756 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:43.756 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:43.907 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:44.197 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:46.733 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:46.734 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:50.017 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:21:46.636 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.049 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:21:46.668 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.076 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:21:46.695 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.084 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:21:46.683 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.087 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:21:46.686 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.091 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:21:46.711 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.159 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:21:46.779 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.184 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:21:46.783 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:50.248 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:21:46.868 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:04:51.348 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:51.601 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:54.477 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:54.477 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:55.122 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:55.122 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:57.360 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:57.360 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:04:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:04:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:04:58.196 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:03.569 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:03.570 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:10.766 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:11.258 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:11.258 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:16.914 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:16.914 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:22.304 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:22.700 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:22.700 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:25.308 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:25.308 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:27.090 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:29.737 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:29.738 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:30.472 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:30.974 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:30.974 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:32.256 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:33.689 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:33.993 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:33.993 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:36.692 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:36.692 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:39.102 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:39.102 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:41.280 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:48.030 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:48.362 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:48.362 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:50.163 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:50.163 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:52.229 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:52.487 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:55.401 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:05:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:05:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:05:57.702 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:01.015 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:01.015 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:03.521 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:03.521 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:05.380 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:05.897 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:05.897 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:08.168 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:09.956 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:10.353 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:12.090 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:15.419 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:15.828 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:15.828 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:17.929 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:19.937 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:19.937 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:21.704 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:21.705 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:23.158 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:23.556 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:23.557 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:26.247 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:26.247 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:26.403 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:23:23.022 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.438 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:23:23.057 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.449 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:23:23.068 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.450 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:23:23.070 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.489 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:23:23.087 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.491 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:23:23.090 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.514 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:23:23.133 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.520 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:23:23.140 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:26.571 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:23:23.170 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:27.835 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:27.835 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:28.246 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:28.585 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:28.585 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:31.127 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:31.127 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:33.339 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:33.884 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:36.180 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:36.180 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:37.063 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:37.064 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:39.994 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:39.994 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:42.407 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:42.407 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:44.925 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:45.749 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:45.994 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:45.994 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:49.535 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:54.494 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:54.495 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:06:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:06:59.435 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:06:59.435 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:02.389 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:02.389 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:04.607 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:04.608 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:05.356 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:05.758 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:05.758 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:07.961 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:07.961 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:09.132 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:09.433 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:11.648 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:13.327 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:16.032 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:16.032 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:18.017 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:18.812 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:19.076 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:23.138 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:24.947 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:25.151 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:25.444 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:25.444 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:27.998 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:29.479 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:29.480 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:32.170 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:32.171 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:33.963 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:34.528 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:37.625 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:37.625 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:38.693 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:39.136 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:40.866 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:41.253 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:41.516 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:41.516 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:43.374 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:43.375 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:43.771 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:44.047 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:44.047 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:47.010 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:47.011 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:47.300 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:47.606 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:47.606 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:50.814 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:53.411 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:53.411 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:53.989 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:54.257 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:56.137 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:56.137 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:07:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:56.568 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:07:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:07:59.964 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:00.439 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:00.719 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:00.719 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:03.907 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:03.907 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:04.790 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:07.674 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:07.674 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:08.598 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:11.235 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:11.235 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:11.605 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:11.879 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:14.224 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:14.224 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:14.990 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:15.280 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:15.280 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:18.772 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:18.772 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:19.683 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:22.251 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:22.252 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:23.129 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:23.413 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:23.413 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:25.848 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:25.848 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:26.869 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:26.869 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:28.455 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:28.455 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:28.943 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:29.316 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:29.316 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:31.047 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:31.048 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:31.641 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:31.952 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:31.952 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:35.134 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:35.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:36.090 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:38.141 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:38.141 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:40.740 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:41.217 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:41.217 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:43.891 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:43.891 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:45.110 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:45.431 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:45.431 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:48.193 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:48.193 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:50.655 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:50.940 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:50.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:53.767 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:53.767 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:08:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:57.112 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:08:59.498 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:08:59.499 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:00.857 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:00.857 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:02.610 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:02.610 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:06.884 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:06.884 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:07.690 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:08.028 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:10.987 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:12.186 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:12.418 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:14.972 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:17.387 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:17.387 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:20.457 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:20.457 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:25.871 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:26.602 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:26.602 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:29.581 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:32.914 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:33.299 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:36.390 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:36.390 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:40.982 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:40.983 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:42.903 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:44.281 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:44.281 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:46.892 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:46.892 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:49.669 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:49.910 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:49.910 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:52.439 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:52.439 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:09:57.025 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:57.025 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:09:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:09:59.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:02.649 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:02.649 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:04.431 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:04.431 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:08.093 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:08.720 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:08.721 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:13.860 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:13.861 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:14.664 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:14.665 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:18.666 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:18.667 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:22.171 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:22.171 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:23.521 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:23.521 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:28.179 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:28.179 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:35.221 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:35.734 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:36.106 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:39.994 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:39.994 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:42.030 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:43.082 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:43.325 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:43.325 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:46.296 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:47.652 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:47.653 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:50.597 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:50.598 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:51.564 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:10:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:10:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:10:59.492 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:04.366 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:04.366 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:05.711 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:05.984 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:05.985 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:09.215 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:10.108 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:14.445 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:14.445 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:15.915 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:20.052 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:20.230 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:20.671 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:20.671 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:24.283 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:25.006 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:25.007 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:28.137 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:28.138 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:29.261 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:34.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:39.843 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:28:36.463 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:39.877 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:28:36.497 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:39.891 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:39.904 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:28:36.524 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:39.905 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:28:36.504 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:39.906 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:28:36.526 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:39.909 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:28:36.508 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:40.097 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:28:36.695 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:40.113 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:28:36.733 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:40.137 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:28:36.753 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:40.151 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:40.152 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:45.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:46.741 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:47.137 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:47.137 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:51.628 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:52.425 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:52.701 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:52.701 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:56.104 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:11:56.360 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:11:56.618 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:11:56.618 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:00.935 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:00.935 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:01.854 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:05.929 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:05.929 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:06.125 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:06.582 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:10.658 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:12.094 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:12.095 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:13.114 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:13.114 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:15.829 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:15.829 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:17.716 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:20.323 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:20.619 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:20.619 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:24.257 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:24.257 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:26.001 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:26.001 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:28.291 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:30.310 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:30.401 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:30.954 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:30.955 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:31.724 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:32.240 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:32.240 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:34.009 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:34.801 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:35.363 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:35.363 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:36.360 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:36.360 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:40.627 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:40.975 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:40.975 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:44.276 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:44.535 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:44.535 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:47.094 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:47.094 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:49.769 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:49.770 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:52.968 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:52.968 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:53.177 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:53.499 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:12:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:12:56.340 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:12:56.341 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:01.081 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:01.081 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:02.637 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:03.209 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:03.209 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:06.486 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:09.769 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:11.423 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:11.423 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:11.720 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:30:08.340 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.727 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:30:08.347 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.738 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:30:08.357 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.740 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:30:08.360 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.771 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:30:08.370 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.786 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:30:08.386 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.811 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:30:08.430 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.813 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:30:08.433 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:11.850 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:30:08.449 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:13:14.260 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:14.261 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:15.900 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:15.900 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:17.920 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:17.921 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:21.587 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:25.431 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:25.432 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:27.404 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:27.405 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:29.812 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:29.813 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:31.870 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:32.200 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:32.201 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:34.288 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:39.616 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:40.134 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:40.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:43.588 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:47.898 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:47.898 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:53.137 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:53.137 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:13:56.217 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:13:56.431 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:13:56.432 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:02.506 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:02.507 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:03.133 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:04.768 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:05.103 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:05.104 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:08.127 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:08.127 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:09.705 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:10.001 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:10.001 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:13.437 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:13.438 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:17.553 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:17.554 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:20.539 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:20.539 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:23.534 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:23.815 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:23.815 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:27.303 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:27.304 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:29.963 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:30.431 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:33.733 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:33.733 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:37.091 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:37.311 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:37.311 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:40.271 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:43.113 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:43.328 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:46.668 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:46.668 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:47.347 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:47.347 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:49.133 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:49.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:50.025 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:50.025 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:51.021 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:51.021 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:52.208 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:52.208 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:53.512 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:53.512 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:53.653 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:14:53.901 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:53.901 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:55.997 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:56.374 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:58.643 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:59.181 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:14:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:14:59.676 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:00.067 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:00.067 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:00.520 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:00.520 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:03.015 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:03.777 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:03.777 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:05.417 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:05.417 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:06.334 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:06.335 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:06.794 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:07.498 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:07.499 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:07.680 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:07.990 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:07.990 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:10.481 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:10.481 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:10.924 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:10.924 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:12.576 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:12.576 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:13.088 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:13.088 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:13.526 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:13.526 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:15.341 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:15.341 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:15.733 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:18.393 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:19.124 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:19.125 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:20.666 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:20.666 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:21.072 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:21.648 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:22.077 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:23.019 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:23.328 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:23.328 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:25.254 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:25.254 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:25.619 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:25.619 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:25.942 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:26.227 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:26.227 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:28.898 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:32.997 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:33.520 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:33.520 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:36.448 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:36.893 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:40.007 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:41.124 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:43.309 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:43.309 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:46.016 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:46.342 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:46.354 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:49.110 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:49.111 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:49.457 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:50.907 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:51.462 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:51.463 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:53.461 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:53.462 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:53.929 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:53.929 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:15:54.478 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:54.478 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:15:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:15:56.477 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:00.273 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:16:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:00.605 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:03.710 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:03.710 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:08.852 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:08.854 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:11.092 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:11.092 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:11.745 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:16:12.212 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:12.212 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:14.177 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:14.177 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:19.320 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:19.321 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:20.986 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:20.986 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:21.455 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:16:21.933 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:21.933 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:24.187 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:28.188 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:29.167 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:29.168 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:30.196 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:30.196 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:30.501 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:16:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:30.770 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:33.415 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:33.416 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:38.649 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:39.930 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:40.699 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:40.699 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:41.048 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:16:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:41.449 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:44.041 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:44.041 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:47.576 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:47.576 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:49.240 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:50.419 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:50.419 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:16:51.160 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:51.160 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:53.043 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:16:59.368 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:16:59.369 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:00.215 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:00.215 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:01.074 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:01.074 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:17:01.824 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:01.824 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:04.442 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:04.442 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:11.768 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:12.793 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:14.067 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:14.067 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:14.925 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:17:15.713 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:15.713 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:17.242 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:17.242 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:23.276 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:23.276 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:24.708 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:24.708 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:25.629 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:25.629 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:26.707 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:17:27.108 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:27.109 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:30.537 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:30.537 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:34.109 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:34.877 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:34.877 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:36.389 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:36.390 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:17:38.451 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:38.452 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:41.699 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:41.700 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:46.996 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:46.997 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:51.929 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:17:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:52.288 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:17:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:17:55.731 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:01.055 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:01.055 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:06.520 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:10.425 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:17.369 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:17.369 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:24.579 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:24.579 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:38.084 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:38.085 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:39.450 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:39.917 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:41.929 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:41.929 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:42.523 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:43.025 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:43.593 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:43.593 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:44.714 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:47.590 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:47.590 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:54.867 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:18:55.609 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:55.610 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:57.299 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:58.979 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:18:59.631 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:18:59.631 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:19:09.345 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:09.345 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:11.398 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:19:11.813 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:11.813 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:15.137 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:19:15.855 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:15.856 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:21.053 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:21.053 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:22.624 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:24.717 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:24.718 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:26.325 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:19:26.639 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:26.639 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:48.163 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:48.164 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:19:49.125 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:19:49.125 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:20:13.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:13.824 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:23.460 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:24.459 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:24.459 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:25.031 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:20:25.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:25.740 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:29.531 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:29.531 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:35.603 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:20:36.331 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:36.332 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:39.527 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:20:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:44.542 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:46.449 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:51.345 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:20:51.700 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:51.700 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:54.358 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:56.768 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:20:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:56.989 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:20:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:20:59.680 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:02.827 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:21:03.082 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:03.082 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:06.648 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:06.648 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:09.533 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:21:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:14.976 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:17.213 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:20.243 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:21:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:20.585 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:26.453 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:28.780 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:30.105 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:30.105 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:30.909 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:21:32.556 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:32.556 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:33.648 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:38:30.268 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.668 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:38:30.266 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.676 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:38:30.296 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.697 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:38:30.317 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.701 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:38:30.300 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.715 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:38:30.335 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.921 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:38:30.541 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.939 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:38:30.558 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:33.964 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:38:30.563 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:34.484 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:34.484 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:38.376 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:38.376 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:41.053 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:41.053 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:21:42.572 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:42.573 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:46.484 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:46.484 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:50.081 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:50.081 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:52.890 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:52.890 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:21:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:21:58.458 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:21:58.459 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:02.346 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:02.350 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:02.508 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:38:59.127 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.521 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:38:59.136 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.526 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:38:59.146 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.544 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:38:59.151 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.545 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:38:59.138 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.566 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:38:59.180 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.566 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:38:59.158 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.576 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:38:59.195 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:02.625 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:38:59.218 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:22:04.910 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:04.911 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:06.936 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:11.834 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:22:11.892 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:22:11.854+0000 b375e700 -1 handle_data failed with exit code -32 2019-12-09T13:22:12.224 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:12.224 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:14.096 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:14.096 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:14.532 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:22:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:14.809 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:18.556 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:22:19.778 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:19.778 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:22.324 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:22.324 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:23.504 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:23.505 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:29.289 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:29.289 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:40.586 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:40.586 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:43.767 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:43.767 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:22:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:22:50.065 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:23:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:23:00.546 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:23:00.546 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:23:04.244 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:23:04.244 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:23:15.254 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:23:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:23:15.598 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:23:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:23:19.041 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:23:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:23:20.654 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:24:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:24:18.379 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:24:27.098 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:24:27.098 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:24:31.403 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:24:31.309+0000 baf6d700 -1 handle_data failed with exit code -104 2019-12-09T13:25:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:25:41.984 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:25:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:25:44.458 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:26:24.085 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:26:24.097 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:14.764 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:44:11.384 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:14.770 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:44:11.388 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:14.788 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:44:11.408 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:14.792 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:44:11.391 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:14.800 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:44:11.397 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:14.801 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:44:11.421 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:14.991 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:44:11.611 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:15.025 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:44:11.644 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:15.045 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:44:11.644 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:38.596 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:27:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:38.834 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:41.111 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:41.112 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:42.316 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:42.316 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:42.563 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:27:42.818 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:42.818 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:45.849 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:45.850 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:51.383 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:51.383 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:27:56.480 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:56.480 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:27:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:27:59.269 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:12.872 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:12.873 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:15.550 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:20.714 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:20.714 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:25.863 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:28.658 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:28.659 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:28.660 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:28.660 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:28:28.660 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:28:28.043+0000 d5fa3700 -1 handle_data failed with exit code -104 2019-12-09T13:28:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:28.875 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:31.965 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:32.155 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:33.952 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:33.952 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:37.177 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:28:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:37.562 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:41.799 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:41.801 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:43.038 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:43.038 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:45.663 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:45.663 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:54.861 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:54.861 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:55.642 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:55.642 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:28:57.214 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:28:57.658 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:28:57.658 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:00.236 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:02.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:16.559 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:16.559 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:21.010 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:21.010 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:25.581 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:25.581 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:26.890 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:26.890 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:27.771 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:29:27.722+0000 ddfb3700 -1 handle_data failed with exit code -104 2019-12-09T13:29:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:28.808 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:43.077 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:43.079 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:48.448 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:48.449 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:54.263 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:55.850 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:55.850 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:29:55.879 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:46:52.497 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:55.966 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:46:52.560 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:55.970 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:46:52.590 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:55.975 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:46:52.574 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:56.053 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:46:52.672 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:56.059 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:46:52.679 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:56.075 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:46:52.695 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:56.129 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:46:52.712 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:56.204 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:46:52.824 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:29:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:29:57.723 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:10.112 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:10.112 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:14.560 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:20.710 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:21.649 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:22.416 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:30:22.365+0000 94f21700 -1 handle_data failed with exit code -104 2019-12-09T13:30:23.833 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:23.834 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:37.253 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:37.254 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:43.097 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:43.097 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:49.940 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:49.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:50.740 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:50.740 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:51.976 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:30:51.963+0000 48272700 -1 handle_data failed with exit code -104 2019-12-09T13:30:52.376 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:30:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:52.671 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:30:54.955 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:30:54.955 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:32:11.179 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:32:11.188 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:32:22.877 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:49:19.497 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:22.885 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:49:19.505 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:22.904 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:49:19.524 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:22.905 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:49:19.504 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:22.909 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:49:19.529 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:22.918 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:49:19.517 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:23.004 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:49:19.623 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:23.052 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:49:19.651 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:23.099 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:49:19.719 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:48.413 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:32:48.414 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:32:50.666 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:32:50.666 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:04.402 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:05.745 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:05.745 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:34:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:45.999 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:52.020 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:52.020 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:52.502 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:34:54.260 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:34:54.262 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:35:25.798 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:35:25.799 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:35:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:35:48.133 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:36:10.680 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:36:10.680 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:36:12.095 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:36:12.095 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:36:28.589 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:36:28.589 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:36:29.400 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:36:29.400 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:36:30.231 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:36:30.167+0000 da7ac700 -1 handle_data failed with exit code -104 2019-12-09T13:37:01.807 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:37:01.807 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:37:16.118 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:37:16.118 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:37:19.795 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:37:19.766+0000 9271c700 -1 handle_data failed with exit code -104 2019-12-09T13:37:21.132 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:37:21.132 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:37:22.358 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:37:22.359 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:37:23.326 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:37:23.235+0000 ab74e700 -1 handle_data failed with exit code -104 2019-12-09T13:37:35.034 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:54:31.654 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.042 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:54:31.662 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.049 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:54:31.670 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.053 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:54:31.673 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.059 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:54:31.658 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.065 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:54:31.664 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.244 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:54:31.864 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.257 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:54:31.877 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:35.295 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:54:31.894 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:58.876 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:37:58.877 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:38:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:38:13.768 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:38:17.609 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:55:14.229 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.661 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:55:14.277 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.661 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:55:14.279 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.661 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:55:14.278 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.686 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:55:14.284 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.688 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:55:14.287 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.704 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:55:14.324 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.709 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:55:14.329 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.758 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:55:14.356 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:17.825 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:38:17.767+0000 f6fe5700 -1 handle_data failed with exit code -104 2019-12-09T13:38:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:38:18.745 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:38:19.644 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:38:19.645 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:38:20.530 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:38:20.479+0000 4b278700 -1 handle_data failed with exit code -104 2019-12-09T13:38:52.543 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:38:52.599 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:38:52.594+0000 10c810700 -1 handle_data failed with exit code -104 2019-12-09T13:38:52.834 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:38:52.834 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:38:55.144 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:38:55.144 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:40:17.824 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:40:17.825 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:40:55.553 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:40:55.554 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:40:57.746 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:40:57.747 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:42:05.395 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:42:05.395 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:42:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:42:07.099 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:42:45.454 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:42:45.455 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:42:48.594 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:59:45.214 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.602 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:59:45.222 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.611 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:59:45.231 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.618 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:59:45.238 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.621 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:59:45.220 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.625 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:59:45.224 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.777 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:03:59:45.397 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.789 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:03:59:45.409 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:48.804 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:03:59:45.403 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:43:58.037 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:43:58.038 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:44:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:44:34.575 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:44:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:44:35.809 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:44:36.955 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:44:36.920+0000 46a6f700 -1 handle_data failed with exit code -104 2019-12-09T13:45:34.198 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:45:34.198 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:45:34.969 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:45:34.969 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:45:35.891 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:45:35.841+0000 12283c700 -1 handle_data failed with exit code -104 2019-12-09T13:46:09.445 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:46:09.445 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:46:39.428 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:03:36.048 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.476 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:03:36.096 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.504 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:03:36.103 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.507 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:03:36.106 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.509 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:03:36.130 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.511 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:03:36.132 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.593 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:03:36.213 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.622 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:03:36.221 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:46:39.624 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:03:36.245 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:47:15.809 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:47:55.491 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:47:55.491 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:47:56.364 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:47:56.365 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:47:56.916 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:47:56.889+0000 47a71700 -1 handle_data failed with exit code -104 2019-12-09T13:47:57.331 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:04:53.951 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.343 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:04:53.962 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.354 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:04:53.974 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.355 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:04:53.954 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.361 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:04:53.982 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.368 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:04:53.967 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.411 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:04:54.031 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.450 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:04:54.071 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:57.468 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:04:54.067 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:03.021 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:49:03.022 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:49:03.946 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:49:03.947 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:49:05.091 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:49:04.836+0000 df7b6700 -1 handle_data failed with exit code -104 2019-12-09T13:49:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:49:40.472 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:51:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:51:11.704 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:51:52.204 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:51:52.204 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:51:54.093 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:51:54.093 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:53:01.891 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:53:01.892 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:53:04.890 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:53:04.890 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:53:10.541 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:10:07.161 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.548 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:10:07.169 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.556 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:10:07.155 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.562 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:10:07.182 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.566 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:10:07.187 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.571 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:10:07.170 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.744 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:10:07.365 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.788 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:10:07.408 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:10.841 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:10:07.439 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:53:45.174 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:55:01.474 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:11:58.094 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.499 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:11:58.119 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.515 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:11:58.135 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.553 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:11:58.173 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.564 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:11:58.185 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.589 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:11:58.188 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.600 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:11:58.198 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.617 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:11:58.238 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:01.787 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:11:58.385 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:55:45.997 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:55:45.998 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:56:20.705 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:56:20.705 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:56:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:56:22.082 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:56:23.153 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:56:23.095+0000 ee7d4700 -1 handle_data failed with exit code -104 2019-12-09T13:57:29.043 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:57:29.043 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:57:30.623 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:57:30.623 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:57:31.911 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T13:57:31.851+0000 bef75700 -1 handle_data failed with exit code -104 2019-12-09T13:58:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T13:58:12.406 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:58:12.406 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:58:14.396 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:58:14.397 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:58:20.711 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:15:17.331 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.718 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:15:17.338 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.726 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:15:17.347 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.731 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:15:17.351 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.732 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:15:17.331 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.738 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:15:17.337 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.865 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:15:17.486 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.887 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:15:17.508 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:20.947 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:15:17.545 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:58:27.887 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:58:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:58:31.282 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:58:51.619 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:58:51.619 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:59:09.660 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:59:09.660 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T13:59:39.519 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T13:59:39.520 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:00:10.073 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:00:10.073 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:00:32.787 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:00:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:00:33.073 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:00:36.785 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:00:36.786 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:01.256 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:06.173 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:06.173 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:10.796 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:12.612 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:13.735 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T14:01:13.696+0000 61ebb700 -1 handle_data failed with exit code -104 2019-12-09T14:01:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:25.240 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:36.857 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:37.844 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:37.844 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:01:39.423 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T14:01:39.385+0000 c4f81700 -1 handle_data failed with exit code -104 2019-12-09T14:01:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:01:53.566 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:02:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:02:06.634 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:02:07.706 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:02:07.706 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:02:08.069 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T14:02:08.053+0000 e97ca700 -1 handle_data failed with exit code -104 2019-12-09T14:02:21.918 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:02:21.919 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:02:46.751 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:02:46.817 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:02:47.888 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:02:47.889 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:02:49.707 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T14:02:49.604+0000 aff57700 -1 handle_data failed with exit code -104 2019-12-09T14:03:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:03:13.825 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:03:23.201 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:20:19.814 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.231 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:20:19.851 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.260 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:20:19.880 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.298 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:20:19.918 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.311 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:20:19.910 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.320 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:20:19.918 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.385 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:20:20.005 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.468 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:20:20.067 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:23.485 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:20:20.106 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.953 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:20:28.574 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.959 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:20:28.580 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.968 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:20:28.588 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.971 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:20:28.592 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.981 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:20:28.580 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.984 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:20:28.583 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:31.989 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:20:28.610 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:32.011 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:20:28.632 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:32.061 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:20:28.660 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:03:32.722 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:03:33.787 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:03:33.787 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:03:35.346 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T14:03:35.278+0000 d87a8700 -1 handle_data failed with exit code -104 2019-12-09T14:04:01.065 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:04:01.065 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:04:15.920 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:04:15.920 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:04:17.268 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:04:17.269 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:04:18.156 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T14:04:18.077+0000 d1f9b700 -1 handle_data failed with exit code -104 2019-12-09T14:04:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:04:42.525 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:04:42.525 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:04:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:04:45.478 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:05:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:05:12.113 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:05:13.813 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:05:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:05:14.089 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:05:16.934 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:05:16.934 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:06:12.139 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:06:12.477 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:06:12.477 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:06:15.065 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:06:15.065 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:06:56.424 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:06:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:06:56.704 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:06:58.800 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:06:58.801 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:21.566 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:21.566 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:23.990 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:07:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:24.236 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:26.370 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:07:28.562 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:28.562 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:31.122 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:31.123 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:43.912 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:07:54.450 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:24:51.070 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.459 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:24:51.080 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.521 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:24:51.141 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.523 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:24:51.143 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.579 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:24:51.177 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.585 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:24:51.180 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.661 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:24:51.281 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.724 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:24:51.344 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:54.748 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:24:51.346 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:07:57.109 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:07:57.109 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:08.821 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:08:15.207 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:15.207 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:19.255 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:24.799 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:08:25.123 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:25.123 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:28.864 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:28.865 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:08:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:33.851 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:39.765 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:08:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:39.980 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:42.344 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:42.344 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:44.643 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:25:41.263 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.651 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:25:41.271 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.657 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:25:41.277 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.660 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:25:41.280 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.662 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:25:41.261 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.669 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:25:41.268 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.714 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:25:41.334 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.717 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:25:41.338 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:44.730 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:25:41.328 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:08:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:46.023 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:49.235 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:54.138 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:55.224 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:55.224 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:56.514 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:08:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:08:58.024 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:00.302 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:00.303 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:01.207 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:01.208 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:02.248 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:03.826 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:03.826 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:04.822 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:07.298 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:08.441 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:08.441 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:09.729 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:09.729 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:10.618 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:10.619 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:11.714 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:11.714 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:13.116 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:13.116 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:14.379 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:14.379 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:16.049 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:16.049 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:16.647 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:17.651 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:18.361 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:18.361 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:19.121 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:19.727 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:20.645 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:21.473 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:21.474 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:23.122 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:23.122 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:24.370 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:25.444 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:25.444 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:26.202 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:26.202 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:27.225 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:27.225 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:27.966 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:28.594 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:28.595 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:29.322 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:29.867 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:29.867 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:09:30.646 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:30.647 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:33.595 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:33.596 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:38.171 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:38.171 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:39.998 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:41.392 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:09:41.691 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:41.691 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:44.741 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:44.741 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:49.631 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:50.176 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:09:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:50.720 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:53.556 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:53.556 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:09:58.057 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:09:58.058 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:02.974 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:02.975 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:10:05.795 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:05.796 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:08.727 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:08.727 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:20.218 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:20.218 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:31.782 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:31.782 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:10:37.739 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:37.739 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:39.942 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:39.942 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:10:54.309 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:10:54.309 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:10.353 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:20.318 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:11:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:20.728 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:25.187 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:25.188 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:27.997 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:31.189 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:31.189 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:34.622 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:34.622 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:39.508 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:39.508 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:11:41.740 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:41.740 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:44.135 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:44.135 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:45.330 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:28:41.950 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.335 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:28:41.955 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.376 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:28:41.996 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.377 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:28:41.998 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.410 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:28:42.007 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.425 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:28:42.023 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.434 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:28:42.054 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.480 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:28:42.100 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:45.610 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:28:42.209 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:11:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:51.979 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:11:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:11:57.340 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:12:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:12:07.425 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:12:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:12:14.089 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:12:20.277 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:12:20.638 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:12:20.638 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:12:22.949 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:12:22.949 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:12:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:12:38.066 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:12:48.706 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:12:48.706 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:06.712 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:18.443 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:29.921 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:13:30.401 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:30.402 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:33.293 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:33.293 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:39.034 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:44.826 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:13:48.100 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:48.100 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:50.318 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:13:50.318 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:13:54.308 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:30:50.928 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.315 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:30:50.935 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.333 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:30:50.953 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.336 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:30:50.957 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.341 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:30:50.939 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.368 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:30:50.966 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.448 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:30:51.068 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.460 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:30:51.080 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.549 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:30:51.143 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:14:05.301 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:14:15.316 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:14:15.317 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:14:21.343 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:14:21.635 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:14:21.635 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:14:23.822 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:14:23.822 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:14:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:14:49.819 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:08.650 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:20.499 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:15:21.112 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:21.112 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:23.606 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:25.838 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:25.838 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:28.283 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:31.286 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:31.287 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:15:33.040 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:33.040 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:35.323 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:35.323 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:15:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:41.656 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:43.849 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:43.850 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:53.565 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:15:59.909 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:15:59.909 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:01.452 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:04.249 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:06.181 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:09.571 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:29.704 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:29.945 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:29.945 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:32.019 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:34.032 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:37.082 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:37.082 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:38.464 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:38.810 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:38.810 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:41.795 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:41.795 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:43.713 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:43.978 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:45.852 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:45.852 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:46.984 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:50.005 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:50.703 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:50.704 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:52.629 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:52.629 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:54.370 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:55.656 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:55.656 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:16:57.277 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:16:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:16:57.699 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:17:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:17:00.210 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:19:06.433 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:36:03.054 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.456 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:36:03.076 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.462 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:36:03.061 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.464 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:36:03.085 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.470 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:36:03.091 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.489 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:36:03.088 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.668 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:36:03.288 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.694 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:36:03.312 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:19:06.777 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:36:03.376 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.172 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:37:03.791 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.221 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:37:03.840 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.224 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:37:03.844 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.238 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:37:03.836 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.257 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:37:03.868 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.258 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:37:03.853 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.281 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:37:03.902 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.294 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:37:03.915 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:07.309 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:37:03.907 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:22:48.485 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:22:48.485 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:22:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:22:52.690 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:23:47.921 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:23:48.329 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:23:48.330 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:23:51.223 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:23:51.224 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:23:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:23:51.836 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:23:54.480 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:23:54.480 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:01.216 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:24:01.892 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:01.892 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:05.037 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:05.038 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:05.511 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:05.511 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:06.940 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:06.940 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:18.136 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:41:14.757 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.148 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:41:14.768 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.160 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:41:14.780 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.166 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:41:14.785 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.166 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:41:14.760 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.166 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:41:14.764 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.319 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:41:14.939 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.327 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:41:14.947 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:18.376 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:41:14.970 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:24.961 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:24:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:25.245 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:29.352 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:29.352 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:29.893 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:29.894 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:30.626 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:32.476 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:24:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:44.424 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:46.546 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:24:48.551 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:24:48.552 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:25:12.630 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:25:12.630 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:25:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:25:35.483 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:26:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:26:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:26:40.940 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:26:43.353 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:26:43.353 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:26:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:26:45.548 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:28:28.954 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:45:25.574 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:28.993 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:45:25.613 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.032 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:45:25.653 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.034 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:45:25.655 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.048 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:45:25.643 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.054 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:45:25.653 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.114 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:45:25.735 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.167 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:45:25.788 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:29.252 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:45:25.851 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:28:50.927 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:28:51.184 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:28:51.184 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:28:54.309 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:28:54.309 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:28:56.075 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:28:56.075 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:29:29.278 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:46:25.899 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.283 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:46:25.904 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.298 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:46:25.918 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.305 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:46:25.926 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.317 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:46:25.916 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.348 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:46:25.968 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.356 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:46:25.955 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.379 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:46:25.999 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:29.444 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:46:26.043 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.407 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:51:38.026 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.430 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:51:38.038 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.431 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:51:38.051 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.433 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:51:38.032 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.436 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:51:38.057 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.443 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:51:38.042 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.616 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:51:38.236 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.659 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:51:38.274 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:41.726 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:51:38.325 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.750 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:53:47.371 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.797 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:53:47.417 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.804 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:53:47.403 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.825 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:53:47.445 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.825 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:53:47.424 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.826 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:53:47.447 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.870 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:53:47.490 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.888 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:53:47.487 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:36:50.905 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:53:47.526 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:37:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:37:05.074 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:37:05.075 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:37:08.373 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:37:08.373 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:37:10.721 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:37:10.722 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:38:16.206 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:38:16.206 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:39:26.987 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:39:26.987 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:39:51.473 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:56:48.093 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.481 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:56:48.101 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.492 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:56:48.113 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.497 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:56:48.097 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.501 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:56:48.122 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.503 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:56:48.102 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.607 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:04:56:48.228 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.643 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:04:56:48.241 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:51.650 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:04:56:48.270 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:40:28.083 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:40:28.084 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:43:57.576 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:43:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:43:57.918 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:44:02.088 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:44:02.088 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:44:04.309 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:44:04.309 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:44:46.493 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:44:46.703 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:44:46.703 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:44:49.529 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:44:49.530 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:44:51.972 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:44:51.972 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:03.564 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:02:00.184 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.572 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:02:00.192 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.581 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:02:00.202 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.586 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:02:00.207 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.589 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:02:00.189 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.598 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:02:00.197 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.791 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:02:00.411 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.817 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:02:00.437 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:03.846 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:02:00.445 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.547 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:02:09.167 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.560 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:02:09.181 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.583 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:02:09.203 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.597 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:02:09.218 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.636 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:02:09.256 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.640 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:02:09.239 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.644 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:02:09.243 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.659 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:02:09.279 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:12.730 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:02:09.329 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:17.011 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:17.011 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:28.151 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:45:28.470 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:28.470 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:30.846 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:30.847 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:33.288 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:33.288 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:50.190 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:45:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:50.504 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:52.503 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:45:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:45:53.690 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:46:18.985 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:18.986 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:21.899 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:23.036 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:23.036 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:46:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:27.655 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:29.828 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:29.828 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:31.374 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:48.192 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:46:48.423 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:48.424 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:51.478 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:51.478 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:46:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:46:53.252 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:07.463 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:47:10.413 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:10.413 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:13.011 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:15.030 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:15.247 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:15.247 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:15.592 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:15.822 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:15.822 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:16.475 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:47:16.475 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:47:18.857 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-53: 2019-12-09T14:47:18.857 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:18.858 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:18.858 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:18.858 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:18.858 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:18.858 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:18.859 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:18.859 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:18.859 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:18.859 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:18.860 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:18.860 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:18.860 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:18.914 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-54: 2019-12-09T14:47:18.914 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:18.914 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:18.914 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:18.914 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:18.915 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:18.916 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:18.916 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:19.153 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-52: 2019-12-09T14:47:19.154 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:19.154 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:19.154 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:19.155 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:19.155 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:19.155 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:19.155 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:19.155 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:19.156 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:19.156 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:19.156 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:19.156 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:19.157 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:19.625 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-51: 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:19.626 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:19.627 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:19.627 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:19.627 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:19.627 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:19.627 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:19.627 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:34.527 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-65: 2019-12-09T14:47:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:34.528 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:34.528 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:34.528 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:34.528 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:34.528 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:34.529 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:34.529 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:34.529 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:34.529 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:34.529 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:34.529 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:34.530 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:34.813 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-61: 2019-12-09T14:47:34.813 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:34.813 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:34.814 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:34.815 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:34.815 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:34.815 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:34.815 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-62: 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:35.286 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:35.287 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:35.287 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:35.287 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:35.287 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:35.287 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-63: 2019-12-09T14:47:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:35.635 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:35.635 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:35.635 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:35.635 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:35.635 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:35.635 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:35.636 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:35.636 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:35.636 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:35.636 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:35.636 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:51.274 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-75: 2019-12-09T14:47:51.274 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:51.274 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:51.274 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:51.274 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:51.275 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:51.276 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:51.276 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-72: 2019-12-09T14:47:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:51.818 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:51.819 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:51.819 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:51.819 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:51.819 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:51.819 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:52.138 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-73: 2019-12-09T14:47:52.138 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:52.138 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:52.138 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:52.138 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:52.139 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:52.140 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:52.140 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:47:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-74: 2019-12-09T14:47:52.321 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:47:52.321 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:47:52.321 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:47:52.321 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:47:52.321 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:47:52.321 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:47:52.322 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:05.623 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-84: 2019-12-09T14:48:05.623 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:05.623 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:05.623 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:05.623 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:05.623 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:05.624 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:05.625 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:05.642 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-85: 2019-12-09T14:48:05.642 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:05.642 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:05.642 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:05.642 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:05.642 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:05.643 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:05.644 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:05.676 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-83: 2019-12-09T14:48:05.676 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:05.676 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:05.676 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:05.676 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:05.677 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:05.678 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:05.678 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:06.870 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-81: 2019-12-09T14:48:06.870 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:06.870 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:06.871 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:06.871 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:06.871 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:06.871 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:06.871 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:06.872 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:06.872 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:06.872 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:06.872 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:06.872 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:06.872 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:21.088 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-92: 2019-12-09T14:48:21.089 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:21.089 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:21.089 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:21.089 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:21.090 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:21.090 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:21.090 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:21.090 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:21.091 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:21.091 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:21.091 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:21.091 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:21.091 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:21.746 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-91: 2019-12-09T14:48:21.747 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:21.747 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:21.747 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:21.747 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:21.747 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:21.747 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:21.748 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:21.962 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-93: 2019-12-09T14:48:21.962 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:21.962 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:21.962 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:21.962 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:21.963 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:21.963 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:21.963 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:21.963 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:21.963 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:21.963 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:21.964 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:21.964 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:21.964 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-94: 2019-12-09T14:48:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:22.038 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:22.038 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:22.039 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:22.040 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:22.040 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:33.482 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:48:34.214 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:34.214 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:39.230 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:42.414 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:42.635 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:42.966 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:43.233 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:44.556 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:44.757 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:45.381 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:46.265 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:46.474 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:48:46.810 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:48:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-103: 2019-12-09T14:48:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:48.732 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:48.732 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:48.732 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:48.733 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:48.733 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:48.733 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:48.733 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:48.734 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:48.734 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:48.734 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:48.734 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:48.797 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-101: 2019-12-09T14:48:48.797 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:48.798 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:48.798 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:48.798 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:48.798 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:48.798 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:48.798 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:48.799 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:48.799 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:48.799 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:48.799 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:48.800 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:48.800 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-105: 2019-12-09T14:48:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:49.235 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:49.236 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:49.236 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:49.236 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:49.237 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:49.237 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:49.237 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:49.237 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:49.237 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:49.238 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:49.238 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:49.238 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-104: 2019-12-09T14:48:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:49.844 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:49.845 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:49.846 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:49.846 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:49.846 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:49.861 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-107: 2019-12-09T14:48:49.861 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:49.861 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:49.862 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:49.863 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:49.863 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:49.863 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:49.863 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:49.884 INFO:teuthology.orchestra.run.smithi059.stderr:Exception in thread Thread-106: 2019-12-09T14:48:49.884 INFO:teuthology.orchestra.run.smithi059.stderr:Traceback (most recent call last): 2019-12-09T14:48:49.884 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T14:48:49.884 INFO:teuthology.orchestra.run.smithi059.stderr: self.run() 2019-12-09T14:48:49.884 INFO:teuthology.orchestra.run.smithi059.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T14:48:49.884 INFO:teuthology.orchestra.run.smithi059.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T14:48:49.885 INFO:teuthology.orchestra.run.smithi059.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T14:48:49.886 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T14:48:53.527 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:05:50.147 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.563 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:05:50.183 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.586 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:05:50.206 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.603 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:05:50.222 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.636 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:05:50.235 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.641 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:05:50.240 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.744 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:05:50.364 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.773 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:05:50.387 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:48:53.792 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:05:50.391 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:02.073 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:02.438 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:02.439 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:04.511 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:07.212 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:07.517 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:07.517 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:10.373 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:10.373 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:13.773 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:14.000 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:17.838 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:17.838 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:19.455 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:19.871 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:19.872 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:22.615 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:22.615 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:23.024 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:23.277 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:23.277 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:26.243 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:26.243 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:27.313 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:27.313 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:29.949 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:31.090 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:34.121 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:34.121 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:35.847 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:36.221 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:36.221 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:39.729 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:40.150 INFO:teuthology.orchestra.run.smithi059.stderr:Check for valid x-amz-expiration header after PUT ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:40.150 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:43.024 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:48.666 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:48.928 INFO:teuthology.orchestra.run.smithi059.stderr:Check for valid x-amz-expiration header on HEAD request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:48.928 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:50.691 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:50.691 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:55.045 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:55.045 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:55.311 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:49:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:Check for valid x-amz-expiration header on GET request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:55.703 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:49:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:49:58.661 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:03.750 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:03.750 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:05.054 INFO:teuthology.orchestra.run.smithi059.stderr:Check x-amz-expiration-header object tags interaction ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:05.054 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:07.448 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:07.449 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:15.371 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:07:11.992 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.379 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:07:11.999 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.387 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:07:12.008 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.390 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:07:12.011 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.392 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:07:11.991 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.400 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:07:11.999 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.466 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:07:12.086 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.484 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:07:12.104 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.549 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:07:12.148 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:15.551 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:15.822 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:15.822 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:20.043 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:21.974 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:22.344 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:22.344 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:25.573 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:29.398 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:29.398 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:31.864 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:34.716 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:35.105 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:35.105 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:38.940 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:38.940 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:41.318 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:41.558 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:44.489 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:46.817 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:47.120 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:47.121 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:50.043 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:52.133 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:52.478 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:52.479 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:54.783 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:54.783 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:50:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:56.477 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:50:59.437 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:50:59.437 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:04.952 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:05.315 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:08.954 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:08.955 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:10.570 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:10.959 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:10.959 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:14.211 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:14.212 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:16.914 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:17.169 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:17.169 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:21.357 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:21.357 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:23.377 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:23.838 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:26.614 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:26.615 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:28.257 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:30.680 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:30.680 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:31.125 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:35.228 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:35.718 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:39.558 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:39.558 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:40.220 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:40.220 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:42.962 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:51:43.377 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:43.377 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:51:46.320 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:51:46.321 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:53:34.660 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:10:31.281 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.666 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:10:31.286 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.683 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:10:31.303 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.705 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:10:31.305 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.714 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:10:31.313 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.721 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:10:31.342 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.780 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:10:31.401 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.805 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:10:31.404 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:34.836 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:10:31.456 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:53:39.078 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:53:39.078 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:54:55.604 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:54:55.604 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:55:25.058 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:12:21.679 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.065 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:12:21.686 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.081 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:12:21.701 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.084 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:12:21.705 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.091 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:12:21.690 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.100 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:12:21.700 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.162 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:12:21.782 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.186 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:12:21.806 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:25.229 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:12:21.828 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:33.326 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:55:33.746 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:55:33.746 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:55:36.266 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:55:36.266 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:42.418 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:56:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:43.549 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:46.242 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:47.484 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:47.485 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:56:48.493 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:48.493 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:50.424 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:50.424 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:56:53.204 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:53.204 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:56.299 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:56.299 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:56:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:56:58.837 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:56:58.837 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:57:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:57:01.462 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:57:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:57:01.834 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:57:01.834 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:57:04.278 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:57:04.278 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:57:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T14:57:05.263 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:57:05.264 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:57:08.060 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:57:08.060 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:59:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:59:00.246 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T14:59:02.500 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T14:59:02.501 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:00:18.009 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:00:18.009 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:00:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:00:21.372 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:00:36.017 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:17:32.637 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.028 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:17:32.649 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.037 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:17:32.657 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.045 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:17:32.643 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.048 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:17:32.669 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.063 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:17:32.662 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.233 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:17:32.853 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.249 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:17:32.870 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:36.291 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:17:32.890 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:01:01.378 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:01:01.378 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:01:03.314 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:01:03.314 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:01:56.256 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:18:52.876 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.294 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:18:52.915 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.301 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:18:52.921 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.303 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:18:52.923 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.354 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:18:52.953 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.364 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:18:52.963 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.368 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:18:52.988 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.372 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:18:52.992 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:01:56.452 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:18:53.051 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:12.822 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:02:13.214 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:02:13.215 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:02:17.260 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:02:17.261 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:03:32.390 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:32.390 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:35.702 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:35.703 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:38.829 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:38.829 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:03:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:40.253 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:43.301 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:43.301 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:03:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:45.043 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:47.508 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:49.940 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:03:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:50.788 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:54.795 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:54.796 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:03:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:03:59.916 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:03:59.916 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:03.267 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:03.267 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:04:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:05.133 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:08.064 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:10.581 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:04:10.924 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:10.925 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:14.166 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:14.166 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:17.268 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:17.268 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:18.985 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:04:19.348 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:19.348 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:22.036 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:22.036 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:24.926 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:24.926 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:04:26.000 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:26.001 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:29.690 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:29.690 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:33.386 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:37.335 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:04:37.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:37.824 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:41.327 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:41.327 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:46.017 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:46.017 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:04:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:48.371 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:51.668 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:51.668 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:04:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:04:54.913 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:01.800 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:01.800 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:02.754 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:05:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:03.421 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:05.755 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:08.297 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:08.298 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:13.957 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:13.958 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:15.093 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:15.093 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:05:16.000 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:16.000 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:19.228 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:19.229 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:29.819 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:29.820 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:31.559 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:05:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:31.857 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:34.362 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:45.198 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:45.198 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:05:46.905 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:46.905 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:47.173 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:22:43.793 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.176 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:22:43.797 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.334 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:22:43.954 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.334 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:22:43.955 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.343 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:22:43.963 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.346 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:22:43.945 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.357 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:22:43.956 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.535 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:22:44.156 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:47.548 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:22:44.147 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:48.855 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:48.855 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:57.801 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:57.801 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:05:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:05:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:05:58.929 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:01.907 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:01.907 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:11.231 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:06:12.421 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:12.421 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:15.914 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:27.416 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:27.417 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:28.957 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:06:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:29.565 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:32.730 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:41.938 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:42.663 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:06:43.524 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:43.524 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:46.314 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:46.314 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:56.040 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:56.040 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:06:57.266 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:06:57.508 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:06:57.508 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:01.146 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:13.415 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:17.299 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:07:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:17.561 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:20.061 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:20.061 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:30.990 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:30.991 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:07:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:35.904 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:40.012 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:42.642 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:07:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:42.943 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:46.240 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:48.630 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:07:49.687 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:49.687 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:53.600 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:53.601 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:07:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:56.325 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:07:59.247 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:07:59.247 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:11.021 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:11.021 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:14.336 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:15.060 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:08:15.372 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:15.373 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:18.098 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:18.098 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:31.489 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:31.489 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:32.749 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:08:35.319 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:35.319 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:41.720 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:41.720 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:54.971 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:54.971 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:08:58.783 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:08:58.783 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:09:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:01.636 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:03.389 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:03.389 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:04.617 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:04.618 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:09.565 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:14.165 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:14.165 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:16.651 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:19.081 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:09:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:19.357 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:22.153 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:09:25.066 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:09:25.066 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:10:17.974 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:27:14.595 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:17.979 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:27:14.600 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.027 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:27:14.648 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.030 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:27:14.650 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.046 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:27:14.644 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.048 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:27:14.647 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.152 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:27:14.772 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.158 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:27:14.778 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:18.161 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:27:14.761 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.819 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:10:32.819 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:10:58.211 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:27:54.832 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.220 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:27:54.840 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.238 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:27:54.858 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.241 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:27:54.840 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.244 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:27:54.865 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.250 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:27:54.849 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.286 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:27:54.907 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.323 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:27:54.944 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:58.345 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:27:54.944 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:48.149 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:11:48.150 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:11:51.002 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:11:51.002 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:00.841 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:13:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:01.145 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:07.688 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:07.688 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:13.407 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:13.407 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:14.679 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:13:15.609 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:15.610 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:22.298 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:22.298 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:29.484 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:32.097 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:13:32.385 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:32.385 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:38.286 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:38.287 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:43.185 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:48.082 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:13:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:48.636 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:53.686 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:54.863 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:54.863 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:13:58.938 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:13:58.938 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:05.090 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:14:06.065 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:06.065 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:09.967 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:09.967 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:11.940 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:11.941 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:16.435 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:16.435 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:19.892 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:14:20.157 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:20.157 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:22.433 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:22.434 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:23.768 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:23.769 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:25.560 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:14:25.984 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:25.984 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:29.411 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:29.412 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:31.463 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:33.914 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:38.734 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:14:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:39.029 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:45.811 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:45.812 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:51.268 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:51.269 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:53.645 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:14:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:53.972 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:14:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:14:59.131 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:14:59.132 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:01.865 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:02.242 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:04.751 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:05.041 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:05.041 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:08.210 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:08.681 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:08.681 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:12.349 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:12.352 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:16.220 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:16.681 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:16.682 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:19.205 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:19.555 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:24.126 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:26.251 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:26.545 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:37.145 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:37.490 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:37.490 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:41.805 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:42.085 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:48.363 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:48.363 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:15:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:15:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:15:59.267 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:16:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:16:05.185 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:16:05.185 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:16:09.077 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:33:05.697 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.091 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:33:05.711 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.108 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:33:05.728 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.113 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:33:05.733 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.115 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:33:05.714 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.121 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:33:05.720 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.312 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:33:05.932 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.328 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:33:05.949 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:09.339 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:33:05.938 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:17.541 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:16:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:16:18.134 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:16:30.892 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:16:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:16:31.198 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:16:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:16:43.892 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:16:43.892 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:16:54.860 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:16:55.239 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:16:55.240 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:17:08.222 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:17:08.717 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:17:08.717 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:17:22.095 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:17:22.394 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:17:22.394 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:17:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:17:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:17:33.016 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:17:36.211 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:17:36.468 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:17:36.469 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:17:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:17:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:17:43.943 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:17:57.093 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:17:57.382 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:17:57.382 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:18:02.187 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:18:02.496 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:18:02.497 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:18:11.795 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:18:12.093 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:18:12.093 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:18:25.645 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:18:26.123 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:18:26.124 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:18:41.812 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:18:42.060 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:18:42.060 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:18:55.471 INFO:teuthology.orchestra.run.smithi059.stderr:ok 2019-12-09T15:18:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-12-09T15:18:55.752 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T15:18:55.752 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T15:21:20.089 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:38:16.709 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.097 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:38:16.717 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.107 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:38:16.705 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.116 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:38:16.715 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.116 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:38:16.736 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.123 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:38:16.743 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.314 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:38:16.934 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.351 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:38:16.950 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:20.352 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:38:16.972 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.692 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:43:28.312 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.717 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:43:28.319 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.717 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:43:28.329 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.717 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:43:28.334 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.722 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:43:28.321 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.729 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:43:28.328 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.893 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:43:28.514 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.918 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:43:28.538 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:31.933 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:43:28.532 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.757 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:43:58.378 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.794 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:43:58.414 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.833 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:43:58.453 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.836 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:43:58.456 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.841 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:43:58.461 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.865 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:43:58.463 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.869 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:43:58.490 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.870 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:43:58.469 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:27:01.922 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:43:58.521 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.592 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:46:19.212 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.631 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:46:19.251 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.681 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:46:19.301 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.683 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:46:19.303 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.696 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:46:19.295 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.698 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:46:19.298 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.763 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:46:19.384 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.816 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:46:19.436 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:22.848 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:46:19.447 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.158 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:48:37.779 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.166 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:48:37.786 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.179 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:48:37.778 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.181 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:48:37.801 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.186 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:48:37.785 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.188 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:48:37.808 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.283 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:48:37.903 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.325 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:48:37.946 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:41.369 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:48:37.968 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.740 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:52:20.360 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.759 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:52:20.366 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.776 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:52:20.396 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.800 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:52:20.421 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.826 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:52:20.425 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.829 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:52:20.428 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.878 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:52:20.498 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.908 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:52:20.528 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:35:23.954 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:52:20.553 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:52.915 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:53:49.536 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:52.925 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:53:49.546 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:52.934 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:53:49.554 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:52.937 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:53:49.536 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:52.944 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:53:49.564 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:52.945 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:53:49.545 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:53.005 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:53:49.626 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:53.033 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:53:49.654 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:53.070 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:53:49.669 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.451 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:59:02.072 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.456 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:59:02.077 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.485 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:59:02.084 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.519 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:59:02.118 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.580 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:59:02.201 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.588 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:59:02.207 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.707 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:05:59:02.327 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.821 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:05:59:02.441 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:05.894 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:05:59:02.493 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.638 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:00:42.258 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.671 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:00:42.292 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.684 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:00:42.304 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.686 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:00:42.306 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.711 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:00:42.310 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.714 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:00:42.313 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.732 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:00:42.353 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.769 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:00:42.389 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:43:45.795 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:00:42.394 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.365 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:04:11.985 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.373 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:04:11.994 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.387 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:04:11.987 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.388 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:04:12.009 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.390 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:04:11.989 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.395 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:04:12.015 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.523 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:04:12.122 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.529 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:04:12.149 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:47:15.535 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:04:12.156 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.434 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:09:04.055 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.444 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:09:04.060 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.486 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:09:04.107 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.497 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:09:04.095 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.497 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:09:04.112 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.501 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:09:04.100 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.610 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:09:04.231 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.620 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:09:04.240 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:07.694 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:09:04.293 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.471 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:09:24.091 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.477 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:09:24.098 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.487 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:09:24.107 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.491 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:09:24.111 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.492 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:09:24.091 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.519 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:09:24.118 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.523 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:09:24.143 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.535 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:09:24.155 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:27.594 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:09:24.193 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.716 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:14:36.336 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.727 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:14:36.347 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.739 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:14:36.360 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.743 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:14:36.363 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.748 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:14:36.347 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.753 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:14:36.353 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:39.937 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:14:36.557 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:40.022 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:14:36.642 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:40.029 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:14:36.628 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.170 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:17:25.790 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.203 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:17:25.823 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.260 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:17:25.880 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.262 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:17:25.882 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.287 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:17:25.886 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.304 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:17:25.903 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.340 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:17:25.961 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.364 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:17:25.979 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:00:29.428 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:17:26.028 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:50.841 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:19:47.462 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:50.861 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:19:47.481 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:50.875 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:19:47.495 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:50.881 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:19:47.501 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:50.897 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:19:47.495 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:50.903 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:19:47.502 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:51.012 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:19:47.633 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:51.041 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:19:47.661 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:51.103 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:19:47.702 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.110 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:25:00.730 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.114 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:25:00.734 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.130 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:25:00.729 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.131 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:25:00.751 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.138 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:25:00.738 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.142 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:25:00.763 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.312 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:25:00.933 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.343 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:25:00.964 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:04.373 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:25:00.973 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:50.978 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:25:47.598 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:50.991 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:25:47.611 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.038 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:25:47.658 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.042 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:25:47.642 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.052 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:25:47.652 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.053 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:25:47.674 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.065 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:25:47.686 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.102 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:25:47.722 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:51.121 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:25:47.720 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.631 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:27:28.251 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.636 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:27:28.256 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.688 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:27:28.308 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.689 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:27:28.310 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.714 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:27:28.334 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.763 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:27:28.362 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.764 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:27:28.384 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.767 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:27:28.366 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:10:31.851 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:27:28.450 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.216 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:30:10.836 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.221 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:30:10.842 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.234 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:30:10.855 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.236 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:30:10.836 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.239 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:30:10.860 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.246 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:30:10.845 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.367 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:30:10.987 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.379 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:30:10.978 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:14.407 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:30:11.028 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.059 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:34:09.680 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.103 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:34:09.723 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.144 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:34:09.742 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.150 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:34:09.749 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.152 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:34:09.772 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.155 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:34:09.775 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.263 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:34:09.884 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.275 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:34:09.874 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:17:13.293 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:34:09.913 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.425 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:35:23.045 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.437 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:35:23.057 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.449 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:35:23.068 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.451 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:35:23.051 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.457 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:35:23.077 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.461 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:35:23.061 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.521 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:35:23.142 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.530 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:35:23.129 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:26.535 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:35:23.154 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.579 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:40:36.199 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.588 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:40:36.208 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.591 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:40:36.211 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.594 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:40:36.214 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.605 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:40:36.203 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.634 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:40:36.233 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.788 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:40:36.409 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.841 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:40:36.461 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:39.920 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:40:36.519 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.864 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:42:31.484 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.919 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:42:31.539 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.943 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:42:31.564 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.953 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:42:31.552 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.961 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:42:31.560 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.967 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:42:31.588 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:34.993 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:42:31.612 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:35.032 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:42:31.653 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:25:35.069 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:42:31.669 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.660 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:45:47.280 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.668 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:45:47.289 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.673 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:45:47.294 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.676 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:45:47.297 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.679 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:45:47.279 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.690 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:45:47.287 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.799 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:45:47.420 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.829 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:45:47.428 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:50.838 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:45:47.459 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.736 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:50:58.356 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.740 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:50:58.360 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.783 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:50:58.403 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.793 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:50:58.413 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.802 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:50:58.401 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.808 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:50:58.407 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.907 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:50:58.527 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.935 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:50:58.555 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.957 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:50:58.556 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.325 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:50:59.924 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.328 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:50:59.927 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.373 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:50:59.972 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.421 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:51:00.041 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.422 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:51:00.043 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.435 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:51:00.054 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.596 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:51:00.216 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.598 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:51:00.218 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:03.618 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:51:00.238 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.171 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:56:11.791 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.177 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:56:11.798 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.187 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:56:11.808 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.190 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:56:11.810 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.201 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:56:11.800 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.209 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:56:11.809 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.392 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:56:12.012 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.399 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:56:12.019 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:15.476 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:56:12.075 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.513 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:59:20.133 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.556 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:59:20.176 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.561 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:59:20.161 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.563 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:59:20.163 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.580 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:59:20.201 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.582 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:59:20.203 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.660 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:06:59:20.281 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.683 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:06:59:20.303 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:42:23.704 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:06:59:20.303 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.347 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:01:19.962 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.499 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:01:19.966 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.499 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:01:19.974 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.499 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:01:19.968 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.499 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:01:20.060 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.500 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:01:19.977 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.500 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:01:19.982 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.500 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:01:20.090 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:23.500 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:01:20.085 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.178 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:06:32.797 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.186 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:06:32.807 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.201 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:06:32.822 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.207 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:06:32.828 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.210 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:06:32.810 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.220 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:06:32.820 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.401 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:06:33.021 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.434 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:06:33.055 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:36.463 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:06:33.063 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.301 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:07:46.921 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.337 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:07:46.958 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.356 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:07:46.977 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.357 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:07:46.978 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.383 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:07:46.983 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.389 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:07:47.010 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.395 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:07:46.995 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.416 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:07:47.036 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:50:50.472 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:07:47.072 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.825 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:08:27.445 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.837 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:08:27.458 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.838 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:08:27.459 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.872 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:08:27.493 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.897 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:08:27.518 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.917 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:08:27.538 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.924 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:08:27.524 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:30.927 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:08:27.527 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:51:31.024 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:08:27.623 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.707 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:11:40.328 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.722 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:11:40.339 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.729 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:11:40.350 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.732 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:11:40.331 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.733 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:11:40.354 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.740 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:11:40.340 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.876 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:11:40.476 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.884 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:11:40.504 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:43.899 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:11:40.517 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.291 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:16:08.912 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.301 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:16:08.921 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.304 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:16:08.925 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.321 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:16:08.942 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.392 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:16:08.992 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.396 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:16:08.996 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.422 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:16:09.042 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.445 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:16:09.066 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:12.640 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:16:09.240 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.867 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:16:50.487 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.876 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:16:50.496 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.883 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:16:50.504 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.888 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:16:50.509 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.901 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:16:50.501 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.921 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:16:50.521 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.929 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:16:50.550 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:53.934 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:16:50.555 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:54.027 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:16:50.627 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.050 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:22:04.671 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.067 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:22:04.679 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.070 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:22:04.670 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.074 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:22:04.674 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.076 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:22:04.696 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.085 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:22:04.706 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.252 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:22:04.873 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.276 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:22:04.876 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:08.300 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:22:04.921 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.148 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:24:35.769 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.190 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:24:35.807 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.190 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:24:35.807 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.207 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:24:35.828 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.218 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:24:35.818 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.220 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:24:35.820 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.296 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:24:35.917 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.308 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:24:35.908 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:39.325 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:24:35.946 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:12.403 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T17:09:12.403 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T17:09:23.966 INFO:teuthology.orchestra.run.smithi059.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi059.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-09T17:09:23.967 INFO:teuthology.orchestra.run.smithi059.stderr: SubjectAltNameWarning, 2019-12-09T17:09:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T17:09:26.822 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T17:09:26.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T17:09:26.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T17:09:26.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T17:09:26.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T17:09:26.823 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T17:09:27.200 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T17:09:27.372 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T17:09:27.970 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T17:09:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T17:09:28.020 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T17:09:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T17:09:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T17:09:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T17:09:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T17:09:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T17:09:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T17:09:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T17:09:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T17:09:28.023 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T17:09:28.023 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T17:09:28.026 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T17:09:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T17:09:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T17:09:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T17:09:28.032 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T17:09:28.033 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T17:09:28.033 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T17:09:28.033 INFO:teuthology.orchestra.run.smithi059.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T17:09:28.060 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T17:09:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T17:09:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T17:09:28.224 INFO:teuthology.orchestra.run.smithi059.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T17:09:28.224 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T17:09:28.224 INFO:teuthology.orchestra.run.smithi059.stderr:---------------------------------------------------------------------- 2019-12-09T17:09:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:Ran 611 tests in 17456.697s 2019-12-09T17:09:28.225 INFO:teuthology.orchestra.run.smithi059.stderr: 2019-12-09T17:09:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:OK (SKIP=75) 2019-12-09T17:09:28.274 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-12-09T17:09:28.275 INFO:teuthology.orchestra.run.smithi059:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-12-09T17:09:28.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T17:09:28.392 INFO:tasks.s3tests:Cleaning up boto... 2019-12-09T17:09:28.393 INFO:teuthology.orchestra.run.smithi059:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T17:09:28.449 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-12-09T17:09:28.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.557+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.592+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.592+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.605+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:28.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.605+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.617+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.617+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.648 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.647+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:28.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.866+0000 7f001c95fec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T17:09:28.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.866+0000 7f001c95fec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T17:09:28.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.866+0000 7f001c95fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:09:28.867 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.866+0000 7f001c95fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:09:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.866+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.880 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.880+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:28.881 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.880+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.894+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.894+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.907+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.907+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.922+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:28.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.922+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.932+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.932+0000 7f001c95fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:09:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.932+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.946+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:28.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.946+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.963+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:28.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.963+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.986 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.985+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:28.987 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.985+0000 7f001c95fec0 20 zonegroup default 2019-12-09T17:09:28.987 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.985+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:28.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.995+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:28.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.995+0000 7f001c95fec0 10 Cannot find current period zone using local zone 2019-12-09T17:09:28.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:28.995+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.001+0000 7f001c95fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.001+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.016+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.016+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.034+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.034+0000 7f001c95fec0 20 zone default found 2019-12-09T17:09:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.034+0000 7f001c95fec0 20 started sync module instance, tier type = 2019-12-09T17:09:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.034+0000 7f001c95fec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T17:09:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=0 2019-12-09T17:09:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=1 2019-12-09T17:09:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=2 2019-12-09T17:09:29.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=3 2019-12-09T17:09:29.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=4 2019-12-09T17:09:29.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=5 2019-12-09T17:09:29.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.389+0000 7f001c95fec0 20 add_watcher() i=6 2019-12-09T17:09:29.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.420+0000 7f001c95fec0 20 add_watcher() i=7 2019-12-09T17:09:29.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.420+0000 7f001c95fec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:09:29.421 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.420+0000 7effda7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.436+0000 7f0009b89700 0 -- >> v2:172.21.15.161:3300/0 conn(0x55860da081e0 msgr2=0x55860da0a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:29.594 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:29.593+0000 7f000a38a700 0 -- 172.21.15.59:0/4184584180 >> [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] conn(0x55860da19da0 msgr2=0x55860da1c1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.838+0000 7eff50ff9700 20 reqs_thread_entry: start 2019-12-09T17:09:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.838+0000 7eff4a7fc700 20 reqs_thread_entry: start 2019-12-09T17:09:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.838+0000 7f001c95fec0 20 init_complete bucket index max shards: 0 2019-12-09T17:09:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.839+0000 7eff48ff9700 20 reqs_thread_entry: start 2019-12-09T17:09:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.839+0000 7f001c95fec0 20 get_system_obj_state: rctx=0x7fff7819c328 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55860da7d920 s->prefetch_data=0 2019-12-09T17:09:30.841 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.839+0000 7f001c95fec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T17:09:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.859+0000 7f001c95fec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T17:09:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.859+0000 7f001c95fec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.859+0000 7f001c95fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T17:09:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.859+0000 7f001c95fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T17:09:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.859+0000 7f001c95fec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T17:09:30.862 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.859+0000 7f001c95fec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:30.886 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.885+0000 7f001c95fec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T17:09:30.886 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.885+0000 7f001c95fec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T17:09:30.886 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.885+0000 7f001c95fec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:30.905 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.904+0000 7f001c95fec0 10 removing key index: BSJMZOXFZWFFSFJGBTNI 2019-12-09T17:09:30.906 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.904+0000 7f001c95fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=170 2019-12-09T17:09:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:30.919+0000 7effeaffd700 10 RGWWatcher::handle_notify() notify_id 597000454666 cookie 94034242500480 notifier 4947 bl.length()=170 2019-12-09T17:09:31.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:31.134+0000 7f001c95fec0 10 removing email index: bar.client.0+test@test.test 2019-12-09T17:09:31.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:31.134+0000 7f001c95fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-12-09T17:09:31.148 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:31.147+0000 7effeaffd700 10 RGWWatcher::handle_notify() notify_id 597000454746 cookie 94034242498608 notifier 4947 bl.length()=178 2019-12-09T17:09:32.414 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.413+0000 7f001c95fec0 10 removing user buckets index 2019-12-09T17:09:32.414 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.413+0000 7f001c95fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-12-09T17:09:32.430 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.429+0000 7effeaffd700 10 RGWWatcher::handle_notify() notify_id 597000454667 cookie 94034242500480 notifier 4947 bl.length()=169 2019-12-09T17:09:32.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.560+0000 7f001c95fec0 10 removing user index: bar.client.0 2019-12-09T17:09:32.562 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.561+0000 7f001c95fec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-12-09T17:09:32.563 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.561+0000 7f001c95fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-12-09T17:09:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.578+0000 7effeaffd700 10 RGWWatcher::handle_notify() notify_id 597000454747 cookie 94034242498608 notifier 4947 bl.length()=161 2019-12-09T17:09:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.771+0000 7f001c95fec0 20 remove_watcher() i=0 2019-12-09T17:09:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.772+0000 7f001c95fec0 2 removed watcher, disabling cache 2019-12-09T17:09:32.821 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.820+0000 7f001c95fec0 20 remove_watcher() i=1 2019-12-09T17:09:32.864 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.862+0000 7f001c95fec0 20 remove_watcher() i=2 2019-12-09T17:09:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:32.932+0000 7f001c95fec0 20 remove_watcher() i=3 2019-12-09T17:09:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.002+0000 7f001c95fec0 20 remove_watcher() i=4 2019-12-09T17:09:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.053+0000 7f001c95fec0 20 remove_watcher() i=5 2019-12-09T17:09:33.120 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.119+0000 7f001c95fec0 20 remove_watcher() i=6 2019-12-09T17:09:33.159 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.158+0000 7f001c95fec0 20 remove_watcher() i=7 2019-12-09T17:09:33.167 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-12-09T17:09:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.294+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.332 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.331+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.333 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.331+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.343 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.342+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.343 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.342+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.351+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.351+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.371+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.617+0000 7fe8376fdec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T17:09:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.617+0000 7fe8376fdec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T17:09:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.617+0000 7fe8376fdec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:09:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.617+0000 7fe8376fdec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:09:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.617+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.637 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.636+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:33.638 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.636+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.654 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.653+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:33.655 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.653+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.667+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:33.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.667+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.682+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.682+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.692 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.691+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.692 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.691+0000 7fe8376fdec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:09:33.693 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.691+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.704+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.705 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.704+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.722 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.721+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:33.723 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.721+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.731+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.731+0000 7fe8376fdec0 20 zonegroup default 2019-12-09T17:09:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.731+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.742+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.743 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.742+0000 7fe8376fdec0 10 Cannot find current period zone using local zone 2019-12-09T17:09:33.744 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.742+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.752+0000 7fe8376fdec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.752+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.784+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:33.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.784+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:33.806 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.805+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:33.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.805+0000 7fe8376fdec0 20 zone default found 2019-12-09T17:09:33.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.805+0000 7fe8376fdec0 20 started sync module instance, tier type = 2019-12-09T17:09:33.807 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:33.805+0000 7fe8376fdec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T17:09:34.115 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.114+0000 7fe8376fdec0 20 add_watcher() i=0 2019-12-09T17:09:34.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.114+0000 7fe8376fdec0 20 add_watcher() i=1 2019-12-09T17:09:34.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.114+0000 7fe8376fdec0 20 add_watcher() i=2 2019-12-09T17:09:34.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.114+0000 7fe8376fdec0 20 add_watcher() i=3 2019-12-09T17:09:34.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.114+0000 7fe8376fdec0 20 add_watcher() i=4 2019-12-09T17:09:34.116 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.114+0000 7fe8376fdec0 20 add_watcher() i=5 2019-12-09T17:09:34.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.116+0000 7fe8376fdec0 20 add_watcher() i=6 2019-12-09T17:09:34.146 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.145+0000 7fe8376fdec0 20 add_watcher() i=7 2019-12-09T17:09:34.147 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.145+0000 7fe8376fdec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:09:34.147 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.145+0000 7fe7f57e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:34.150 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.149+0000 7fe825128700 0 -- 172.21.15.59:0/660027033 >> v1:172.21.15.59:6789/0 conn(0x55feb421d830 legacy=0x55feb4220e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:34.193 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:34.191+0000 7fe824126700 0 -- 172.21.15.59:0/2516403100 >> [v2:172.21.15.161:6800/9941,v1:172.21.15.161:6801/9941] conn(0x55feb42380c0 msgr2=0x55feb423a4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.913+0000 7fe7677fe700 20 reqs_thread_entry: start 2019-12-09T17:09:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.914+0000 7fe7657fa700 20 reqs_thread_entry: start 2019-12-09T17:09:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.914+0000 7fe8376fdec0 20 init_complete bucket index max shards: 0 2019-12-09T17:09:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.915+0000 7fe7537fe700 20 reqs_thread_entry: start 2019-12-09T17:09:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.915+0000 7fe8376fdec0 20 get_system_obj_state: rctx=0x7fffc9926578 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55feb429bfb0 s->prefetch_data=0 2019-12-09T17:09:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.915+0000 7fe8376fdec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T17:09:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.941+0000 7fe8376fdec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T17:09:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.941+0000 7fe8376fdec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.941+0000 7fe8376fdec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T17:09:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.941+0000 7fe8376fdec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T17:09:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.941+0000 7fe8376fdec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T17:09:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.941+0000 7fe8376fdec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:35.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.956+0000 7fe8376fdec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T17:09:35.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.956+0000 7fe8376fdec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T17:09:35.958 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.956+0000 7fe8376fdec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:35.990 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.989+0000 7fe8376fdec0 10 removing key index: SWOKOQZFVMPIFBJDJCXH 2019-12-09T17:09:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:35.989+0000 7fe8376fdec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=170 2019-12-09T17:09:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.017+0000 7fe80a7fc700 10 RGWWatcher::handle_notify() notify_id 597000454221 cookie 94552432104320 notifier 4959 bl.length()=170 2019-12-09T17:09:36.191 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.190+0000 7fe8376fdec0 10 removing email index: foo.client.0+test@test.test 2019-12-09T17:09:36.191 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.190+0000 7fe8376fdec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-12-09T17:09:36.209 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.208+0000 7fe80a7fc700 10 RGWWatcher::handle_notify() notify_id 597000454186 cookie 94552432119840 notifier 4959 bl.length()=178 2019-12-09T17:09:36.627 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.625+0000 7fe8376fdec0 10 removing user buckets index 2019-12-09T17:09:36.628 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.625+0000 7fe8376fdec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-12-09T17:09:36.640 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.639+0000 7fe80a7fc700 10 RGWWatcher::handle_notify() notify_id 597000454748 cookie 94552432106432 notifier 4959 bl.length()=169 2019-12-09T17:09:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.960+0000 7fe8376fdec0 10 removing user index: foo.client.0 2019-12-09T17:09:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.960+0000 7fe8376fdec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-12-09T17:09:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.960+0000 7fe8376fdec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-12-09T17:09:36.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:36.972+0000 7fe80a7fc700 10 RGWWatcher::handle_notify() notify_id 597000454222 cookie 94552432104320 notifier 4959 bl.length()=161 2019-12-09T17:09:37.135 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.134+0000 7fe8376fdec0 20 remove_watcher() i=0 2019-12-09T17:09:37.136 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.134+0000 7fe8376fdec0 2 removed watcher, disabling cache 2019-12-09T17:09:37.171 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.170+0000 7fe8376fdec0 20 remove_watcher() i=1 2019-12-09T17:09:37.220 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.219+0000 7fe8376fdec0 20 remove_watcher() i=2 2019-12-09T17:09:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.279+0000 7fe8376fdec0 20 remove_watcher() i=3 2019-12-09T17:09:37.338 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.337+0000 7fe8376fdec0 20 remove_watcher() i=4 2019-12-09T17:09:37.374 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.366+0000 7fe8376fdec0 20 remove_watcher() i=5 2019-12-09T17:09:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.428+0000 7fe8376fdec0 20 remove_watcher() i=6 2019-12-09T17:09:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.473+0000 7fe8376fdec0 20 remove_watcher() i=7 2019-12-09T17:09:37.484 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-12-09T17:09:37.609 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.608+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.642+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.643 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.642+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.652 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.651+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.652 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.651+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.667+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.667+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.684 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.683+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.864+0000 7f43ea3d6ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T17:09:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.864+0000 7f43ea3d6ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T17:09:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.864+0000 7f43ea3d6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:09:37.866 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.864+0000 7f43ea3d6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:09:37.866 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.864+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.876+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.876+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.892+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.892+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.906+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.906+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.918 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.914+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:37.918 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.914+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.922+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.922+0000 7f43ea3d6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:09:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.922+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.936+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.938 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.936+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.946+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.946+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.959+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.959+0000 7f43ea3d6ec0 20 zonegroup default 2019-12-09T17:09:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.959+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.966 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.965+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.967 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.965+0000 7f43ea3d6ec0 10 Cannot find current period zone using local zone 2019-12-09T17:09:37.967 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.965+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.972+0000 7f43ea3d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:37.973 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.972+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.980+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.980+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:37.991 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.990+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:37.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.990+0000 7f43ea3d6ec0 20 zone default found 2019-12-09T17:09:37.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.990+0000 7f43ea3d6ec0 20 started sync module instance, tier type = 2019-12-09T17:09:37.992 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:37.990+0000 7f43ea3d6ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T17:09:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43ea3d6ec0 20 add_watcher() i=0 2019-12-09T17:09:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43ea3d6ec0 20 add_watcher() i=1 2019-12-09T17:09:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43ea3d6ec0 20 add_watcher() i=2 2019-12-09T17:09:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43ea3d6ec0 20 add_watcher() i=3 2019-12-09T17:09:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43ea3d6ec0 20 add_watcher() i=4 2019-12-09T17:09:38.409 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43ea3d6ec0 20 add_watcher() i=5 2019-12-09T17:09:38.410 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.407+0000 7f43d7600700 0 -- 172.21.15.59:0/4242316284 >> [v2:172.21.15.59:6802/9847,v1:172.21.15.59:6803/9847] conn(0x7f43cc015190 msgr2=0x7f43cc0155a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T17:09:38.411 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.410+0000 7f43ea3d6ec0 20 add_watcher() i=6 2019-12-09T17:09:38.451 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.450+0000 7f43ea3d6ec0 20 add_watcher() i=7 2019-12-09T17:09:38.451 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.450+0000 7f43ea3d6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:09:38.452 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.450+0000 7f43a7fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:38.469 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:38.467+0000 7f43d7e01700 0 -- 172.21.15.59:0/1750652534 >> v1:172.21.15.161:6789/0 conn(0x55e551b3e4d0 legacy=0x55e551b3e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T17:09:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.166+0000 7f43167fc700 20 reqs_thread_entry: start 2019-12-09T17:09:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.166+0000 7f430ffff700 20 reqs_thread_entry: start 2019-12-09T17:09:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.166+0000 7f43ea3d6ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:09:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.167+0000 7f430e7fc700 20 reqs_thread_entry: start 2019-12-09T17:09:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.167+0000 7f43ea3d6ec0 20 get_system_obj_state: rctx=0x7ffc6663b918 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55e551badfe0 s->prefetch_data=0 2019-12-09T17:09:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.167+0000 7f43ea3d6ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T17:09:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.186+0000 7f43ea3d6ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T17:09:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.186+0000 7f43ea3d6ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T17:09:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.186+0000 7f43ea3d6ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T17:09:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.186+0000 7f43ea3d6ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T17:09:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.186+0000 7f43ea3d6ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T17:09:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.186+0000 7f43ea3d6ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.205+0000 7f43ea3d6ec0 20 rados_obj.operate() r=0 bl.length=505 2019-12-09T17:09:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.205+0000 7f43ea3d6ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T17:09:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.205+0000 7f43ea3d6ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T17:09:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.210+0000 7f43ea3d6ec0 10 removing key index: KDNUNQRQMUHWVVRWJHLM 2019-12-09T17:09:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.210+0000 7f43ea3d6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-12-09T17:09:40.221 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.220+0000 7f43bcff9700 10 RGWWatcher::handle_notify() notify_id 597000454749 cookie 94443406533072 notifier 4706 bl.length()=170 2019-12-09T17:09:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.666+0000 7f43ea3d6ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-12-09T17:09:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.666+0000 7f43ea3d6ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-12-09T17:09:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:40.679+0000 7f43bcff9700 10 RGWWatcher::handle_notify() notify_id 597000454223 cookie 94443406541536 notifier 4706 bl.length()=193 2019-12-09T17:09:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.069+0000 7f43ea3d6ec0 10 removing user buckets index 2019-12-09T17:09:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.069+0000 7f43ea3d6ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-12-09T17:09:41.089 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.088+0000 7f43bcff9700 10 RGWWatcher::handle_notify() notify_id 597000454187 cookie 94443406539040 notifier 4706 bl.length()=184 2019-12-09T17:09:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.835+0000 7f43ea3d6ec0 10 removing user index: testx$tenanteduser.client.0 2019-12-09T17:09:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.835+0000 7f43ea3d6ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-12-09T17:09:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.835+0000 7f43ea3d6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-12-09T17:09:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:41.855+0000 7f43bcff9700 10 RGWWatcher::handle_notify() notify_id 597000454750 cookie 94443406533072 notifier 4706 bl.length()=176 2019-12-09T17:09:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.269+0000 7f43ea3d6ec0 20 remove_watcher() i=0 2019-12-09T17:09:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.269+0000 7f43ea3d6ec0 2 removed watcher, disabling cache 2019-12-09T17:09:42.336 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.335+0000 7f43ea3d6ec0 20 remove_watcher() i=1 2019-12-09T17:09:42.406 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.405+0000 7f43ea3d6ec0 20 remove_watcher() i=2 2019-12-09T17:09:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.473+0000 7f43ea3d6ec0 20 remove_watcher() i=3 2019-12-09T17:09:42.523 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.522+0000 7f43ea3d6ec0 20 remove_watcher() i=4 2019-12-09T17:09:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.591+0000 7f43ea3d6ec0 20 remove_watcher() i=5 2019-12-09T17:09:42.644 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.643+0000 7f43ea3d6ec0 20 remove_watcher() i=6 2019-12-09T17:09:42.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:42.684+0000 7f43ea3d6ec0 20 remove_watcher() i=7 2019-12-09T17:09:42.693 INFO:tasks.s3tests:Removing s3-tests... 2019-12-09T17:09:42.694 INFO:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-12-09T17:09:42.833 INFO:teuthology.run_tasks:Running task swift... 2019-12-09T17:09:42.857 INFO:tasks.swift:clients=['client.0'] 2019-12-09T17:09:42.857 INFO:tasks.swift:Downloading swift... 2019-12-09T17:09:42.857 INFO:teuthology.orchestra.run.smithi059:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-12-09T17:09:42.967 INFO:teuthology.orchestra.run.smithi059.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-12-09T17:09:43.391 INFO:tasks.swift:Creating rgw users... 2019-12-09T17:09:43.391 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret 3b47bqlS4ggKhjlJpVXnwarIAbRkOY2RvEJVNNiYFCnR4K5qlX7kfA== --email foo.client.0+test@test.test --key-type swift --access full 2019-12-09T17:09:43.566 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.565+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.606+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:43.608 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.606+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.616 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.615+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:43.617 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.615+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.626 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.625+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:43.626 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.625+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.647 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.646+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:43.893 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.892+0000 7f9c64b69ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T17:09:43.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.892+0000 7f9c64b69ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T17:09:43.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.892+0000 7f9c64b69ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:09:43.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.892+0000 7f9c64b69ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:09:43.894 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.892+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.908 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.907+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:43.908 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.907+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.922 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.921+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.921+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.935+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.935+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.949+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:43.950 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.949+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.956+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:43.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.956+0000 7f9c64b69ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:09:43.957 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.956+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.964+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:43.965 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.964+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.979+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.979+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:43.995 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.994+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.994+0000 7f9c64b69ec0 20 zonegroup default 2019-12-09T17:09:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:43.994+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.003+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.003+0000 7f9c64b69ec0 10 Cannot find current period zone using local zone 2019-12-09T17:09:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.003+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:44.032 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.030+0000 7f9c64b69ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:44.032 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.030+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:44.044 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.043+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:44.045 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.043+0000 7f9c64b69ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.057+0000 7f9c64b69ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.057+0000 7f9c64b69ec0 20 zone default found 2019-12-09T17:09:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.057+0000 7f9c64b69ec0 20 started sync module instance, tier type = 2019-12-09T17:09:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.057+0000 7f9c64b69ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T17:09:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=0 2019-12-09T17:09:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=1 2019-12-09T17:09:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=2 2019-12-09T17:09:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=3 2019-12-09T17:09:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=4 2019-12-09T17:09:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=5 2019-12-09T17:09:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.360+0000 7f9c64b69ec0 20 add_watcher() i=6 2019-12-09T17:09:44.391 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.390+0000 7f9c64b69ec0 20 add_watcher() i=7 2019-12-09T17:09:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.390+0000 7f9c64b69ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:09:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.390+0000 7f9c227dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:44.397 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.396+0000 7f9c51d93700 0 -- 172.21.15.59:0/2183269119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55d17858cf40 msgr2=0x55d17858f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T17:09:44.599 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.598+0000 7f9c51d93700 0 --2- 172.21.15.59:0/2183269119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55d17858cf40 0x55d17858f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T17:09:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:44.644+0000 7f9c52594700 0 -- 172.21.15.59:0/3364740539 >> [v2:172.21.15.59:6800/8893,v1:172.21.15.59:6801/8893] conn(0x7f9b94022490 msgr2=0x7f9b94024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.280+0000 7f9b98ff9700 20 reqs_thread_entry: start 2019-12-09T17:09:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.281+0000 7f9b927fc700 20 reqs_thread_entry: start 2019-12-09T17:09:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.281+0000 7f9c64b69ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:09:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.282+0000 7f9b90ff9700 20 reqs_thread_entry: start 2019-12-09T17:09:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.282+0000 7f9c64b69ec0 20 get_system_obj_state: rctx=0x7ffccb6c66d8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55d17860a620 s->prefetch_data=0 2019-12-09T17:09:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.282+0000 7f9c64b69ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T17:09:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.293+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T17:09:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.293+0000 7f9c64b69ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.293+0000 7f9c64b69ec0 20 get_system_obj_state: rctx=0x7ffccb6c66e8 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x55d17860c3b0 s->prefetch_data=0 2019-12-09T17:09:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.293+0000 7f9c64b69ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T17:09:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.300+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T17:09:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.300+0000 7f9c64b69ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T17:09:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.300+0000 7f9c64b69ec0 20 get_system_obj_state: rctx=0x7ffccb6c62f8 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x55d17860c3b0 s->prefetch_data=0 2019-12-09T17:09:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.300+0000 7f9c64b69ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-12-09T17:09:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.308+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T17:09:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.308+0000 7f9c64b69ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:09:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.308+0000 7f9c64b69ec0 20 get_system_obj_state: rctx=0x7ffccb6c6478 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x55d17860c660 s->prefetch_data=0 2019-12-09T17:09:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.308+0000 7f9c64b69ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-12-09T17:09:46.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.320+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T17:09:46.321 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.320+0000 7f9c64b69ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:09:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.369+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:09:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.369+0000 7f9c64b69ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.369+0000 7f9c64b69ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-12-09T17:09:46.395 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.393+0000 7f9c32ffd700 10 RGWWatcher::handle_notify() notify_id 597000454224 cookie 94358155531488 notifier 4721 bl.length()=612 2019-12-09T17:09:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.393+0000 7f9c32ffd700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:09:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:46.393+0000 7f9c32ffd700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:47.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.008+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T17:09:47.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.008+0000 7f9c64b69ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T17:09:47.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.008+0000 7f9c64b69ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T17:09:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.027+0000 7f9c32ffd700 10 RGWWatcher::handle_notify() notify_id 597000454188 cookie 94358155546336 notifier 4721 bl.length()=194 2019-12-09T17:09:47.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.027+0000 7f9c32ffd700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T17:09:47.035 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.027+0000 7f9c32ffd700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T17:09:47.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.460+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T17:09:47.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.460+0000 7f9c64b69ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:09:47.463 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.460+0000 7f9c64b69ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-12-09T17:09:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.480+0000 7f9c32ffd700 10 RGWWatcher::handle_notify() notify_id 597000454751 cookie 94358155534864 notifier 4721 bl.length()=183 2019-12-09T17:09:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.480+0000 7f9c32ffd700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T17:09:47.483 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:47.480+0000 7f9c32ffd700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:09:48.060 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.058+0000 7f9c64b69ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:09:48.061 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.058+0000 7f9c64b69ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:48.061 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.058+0000 7f9c64b69ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-12-09T17:09:48.075 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.073+0000 7f9c32ffd700 10 RGWWatcher::handle_notify() notify_id 597000454225 cookie 94358155531488 notifier 4721 bl.length()=636 2019-12-09T17:09:48.075 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.073+0000 7f9c32ffd700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:09:48.076 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.073+0000 7f9c32ffd700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:09:48.450 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T17:09:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "foo.client.0", 2019-12-09T17:09:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. foo.client.0 foo", 2019-12-09T17:09:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T17:09:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T17:09:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [ 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "id": "foo.client.0:foo", 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "permissions": "full-control" 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T17:09:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [], 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [ 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "foo.client.0:foo", 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "3b47bqlS4ggKhjlJpVXnwarIAbRkOY2RvEJVNNiYFCnR4K5qlX7kfA==" 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T17:09:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T17:09:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T17:09:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T17:09:48.455 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T17:09:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T17:09:48.500 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.498+0000 7f9c64b69ec0 20 remove_watcher() i=0 2019-12-09T17:09:48.500 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.498+0000 7f9c64b69ec0 2 removed watcher, disabling cache 2019-12-09T17:09:48.564 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.562+0000 7f9c64b69ec0 20 remove_watcher() i=1 2019-12-09T17:09:48.630 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.628+0000 7f9c64b69ec0 20 remove_watcher() i=2 2019-12-09T17:09:48.672 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.670+0000 7f9c64b69ec0 20 remove_watcher() i=3 2019-12-09T17:09:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.716+0000 7f9c64b69ec0 20 remove_watcher() i=4 2019-12-09T17:09:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.775+0000 7f9c64b69ec0 20 remove_watcher() i=5 2019-12-09T17:09:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.834+0000 7f9c64b69ec0 20 remove_watcher() i=6 2019-12-09T17:09:48.880 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:48.878+0000 7f9c64b69ec0 20 remove_watcher() i=7 2019-12-09T17:09:48.888 INFO:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret qw0Rzekv+JPcIq6eJsfHKXmLiHt270WHwC/fuEYNkfeCiS6smF4qJw== --email bar.client.0+test@test.test --key-type swift --access full 2019-12-09T17:09:49.052 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.050+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.090+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.092 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.090+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.102+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.102+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.112+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.114 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.112+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.137 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.135+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:49.374 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.372+0000 7f417e950ec0 20 RGWRados::pool_iterate: got zonegroup_info.3d1455e7-8faf-4a8c-8789-76d5648a0e74 2019-12-09T17:09:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.372+0000 7f417e950ec0 20 RGWRados::pool_iterate: got zone_info.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf 2019-12-09T17:09:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.372+0000 7f417e950ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:09:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.372+0000 7f417e950ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:09:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.372+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.387+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.387+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.409+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.409+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.430 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.428+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:49.431 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.428+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.449 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.447+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:49.449 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.447+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.459+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.459+0000 7f417e950ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:09:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.459+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.470+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.470+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.488+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:49.490 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.488+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.498+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:09:49.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.498+0000 7f417e950ec0 20 zonegroup default 2019-12-09T17:09:49.501 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.498+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.512 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.510+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.512 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.510+0000 7f417e950ec0 10 Cannot find current period zone using local zone 2019-12-09T17:09:49.512 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.510+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.520+0000 7f417e950ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:09:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.520+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.534+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:09:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.534+0000 7f417e950ec0 20 rados->read ofs=0 len=0 2019-12-09T17:09:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.555+0000 7f417e950ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:09:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.555+0000 7f417e950ec0 20 zone default found 2019-12-09T17:09:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.555+0000 7f417e950ec0 20 started sync module instance, tier type = 2019-12-09T17:09:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.555+0000 7f417e950ec0 20 started zone id=478c2bee-74bb-4cd4-b56a-b2f2fa8233cf (name=default) with tier type = 2019-12-09T17:09:49.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=0 2019-12-09T17:09:49.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=1 2019-12-09T17:09:49.930 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=2 2019-12-09T17:09:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=3 2019-12-09T17:09:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=4 2019-12-09T17:09:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=5 2019-12-09T17:09:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.928+0000 7f417e950ec0 20 add_watcher() i=6 2019-12-09T17:09:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.998+0000 7f416bb7a700 0 -- 172.21.15.59:0/3152105963 >> [v2:172.21.15.59:6802/9847,v1:172.21.15.59:6803/9847] conn(0x7f41640080f0 msgr2=0x7f416400c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.998+0000 7f417e950ec0 20 add_watcher() i=7 2019-12-09T17:09:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.998+0000 7f417e950ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:09:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:49.998+0000 7f40bffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:50.030+0000 7f416b379700 0 -- 172.21.15.59:0/3156480161 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564a266a4a70 msgr2=0x564a266a6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:09:51.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.764+0000 7f40aa7fc700 20 reqs_thread_entry: start 2019-12-09T17:09:51.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.764+0000 7f40a3fff700 20 reqs_thread_entry: start 2019-12-09T17:09:51.766 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.764+0000 7f417e950ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:09:51.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.765+0000 7f40a27fc700 20 reqs_thread_entry: start 2019-12-09T17:09:51.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.765+0000 7f417e950ec0 20 get_system_obj_state: rctx=0x7ffd850d6b88 obj=default.rgw.meta:users.uid:bar.client.0 state=0x564a26719260 s->prefetch_data=0 2019-12-09T17:09:51.767 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.765+0000 7f417e950ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T17:09:51.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.774+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T17:09:51.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.774+0000 7f417e950ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:51.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.774+0000 7f417e950ec0 20 get_system_obj_state: rctx=0x7ffd850d6b98 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x564a2671aff0 s->prefetch_data=0 2019-12-09T17:09:51.777 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.774+0000 7f417e950ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T17:09:51.784 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.782+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T17:09:51.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.782+0000 7f417e950ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T17:09:51.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.782+0000 7f417e950ec0 20 get_system_obj_state: rctx=0x7ffd850d67a8 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x564a2671aff0 s->prefetch_data=0 2019-12-09T17:09:51.785 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.782+0000 7f417e950ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-12-09T17:09:51.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.789+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T17:09:51.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.789+0000 7f417e950ec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:09:51.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.789+0000 7f417e950ec0 20 get_system_obj_state: rctx=0x7ffd850d6928 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x564a2671b2a0 s->prefetch_data=0 2019-12-09T17:09:51.791 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.789+0000 7f417e950ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-12-09T17:09:51.796 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.794+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T17:09:51.796 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.794+0000 7f417e950ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:09:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.863+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:09:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.863+0000 7f417e950ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.863+0000 7f417e950ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-12-09T17:09:51.879 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.877+0000 7f41617fa700 10 RGWWatcher::handle_notify() notify_id 597000454752 cookie 94876471950656 notifier 5013 bl.length()=612 2019-12-09T17:09:51.880 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.877+0000 7f41617fa700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:09:51.880 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:51.877+0000 7f41617fa700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:52.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.115+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T17:09:52.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.115+0000 7f417e950ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T17:09:52.117 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.115+0000 7f417e950ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T17:09:52.133 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.131+0000 7f41617fa700 10 RGWWatcher::handle_notify() notify_id 597000454753 cookie 94876471950656 notifier 5013 bl.length()=194 2019-12-09T17:09:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.131+0000 7f41617fa700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T17:09:52.134 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.131+0000 7f41617fa700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T17:09:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.661+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T17:09:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.661+0000 7f417e950ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:09:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.661+0000 7f417e950ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-12-09T17:09:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.683+0000 7f41617fa700 10 RGWWatcher::handle_notify() notify_id 597000454754 cookie 94876471959904 notifier 5013 bl.length()=183 2019-12-09T17:09:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.683+0000 7f41617fa700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T17:09:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:52.683+0000 7f41617fa700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:09:53.011 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.009+0000 7f417e950ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:09:53.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.009+0000 7f417e950ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:53.012 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.009+0000 7f417e950ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-12-09T17:09:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.029+0000 7f41617fa700 10 RGWWatcher::handle_notify() notify_id 597000454755 cookie 94876471950656 notifier 5013 bl.length()=636 2019-12-09T17:09:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.029+0000 7f41617fa700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:09:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.029+0000 7f41617fa700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:09:53.082 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2019-12-09T17:09:53.082 INFO:teuthology.orchestra.run.smithi059.stdout: "user_id": "bar.client.0", 2019-12-09T17:09:53.082 INFO:teuthology.orchestra.run.smithi059.stdout: "display_name": "Mr. bar.client.0 bar", 2019-12-09T17:09:53.082 INFO:teuthology.orchestra.run.smithi059.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T17:09:53.083 INFO:teuthology.orchestra.run.smithi059.stdout: "suspended": 0, 2019-12-09T17:09:53.083 INFO:teuthology.orchestra.run.smithi059.stdout: "max_buckets": 1000, 2019-12-09T17:09:53.083 INFO:teuthology.orchestra.run.smithi059.stdout: "subusers": [ 2019-12-09T17:09:53.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T17:09:53.084 INFO:teuthology.orchestra.run.smithi059.stdout: "id": "bar.client.0:bar", 2019-12-09T17:09:53.084 INFO:teuthology.orchestra.run.smithi059.stdout: "permissions": "full-control" 2019-12-09T17:09:53.084 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T17:09:53.084 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T17:09:53.085 INFO:teuthology.orchestra.run.smithi059.stdout: "keys": [], 2019-12-09T17:09:53.085 INFO:teuthology.orchestra.run.smithi059.stdout: "swift_keys": [ 2019-12-09T17:09:53.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2019-12-09T17:09:53.085 INFO:teuthology.orchestra.run.smithi059.stdout: "user": "bar.client.0:bar", 2019-12-09T17:09:53.086 INFO:teuthology.orchestra.run.smithi059.stdout: "secret_key": "qw0Rzekv+JPcIq6eJsfHKXmLiHt270WHwC/fuEYNkfeCiS6smF4qJw==" 2019-12-09T17:09:53.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2019-12-09T17:09:53.086 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2019-12-09T17:09:53.086 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": [], 2019-12-09T17:09:53.086 INFO:teuthology.orchestra.run.smithi059.stdout: "op_mask": "read, write, delete", 2019-12-09T17:09:53.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_placement": "", 2019-12-09T17:09:53.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_storage_class": "", 2019-12-09T17:09:53.087 INFO:teuthology.orchestra.run.smithi059.stdout: "placement_tags": [], 2019-12-09T17:09:53.087 INFO:teuthology.orchestra.run.smithi059.stdout: "bucket_quota": { 2019-12-09T17:09:53.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T17:09:53.088 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T17:09:53.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T17:09:53.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T17:09:53.089 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T17:09:53.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T17:09:53.089 INFO:teuthology.orchestra.run.smithi059.stdout: "user_quota": { 2019-12-09T17:09:53.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": false, 2019-12-09T17:09:53.090 INFO:teuthology.orchestra.run.smithi059.stdout: "check_on_raw": false, 2019-12-09T17:09:53.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size": -1, 2019-12-09T17:09:53.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max_size_kb": 0, 2019-12-09T17:09:53.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max_objects": -1 2019-12-09T17:09:53.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2019-12-09T17:09:53.091 INFO:teuthology.orchestra.run.smithi059.stdout: "temp_url_keys": [], 2019-12-09T17:09:53.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "rgw", 2019-12-09T17:09:53.091 INFO:teuthology.orchestra.run.smithi059.stdout: "mfa_ids": [] 2019-12-09T17:09:53.091 INFO:teuthology.orchestra.run.smithi059.stdout:} 2019-12-09T17:09:53.092 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T17:09:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.129+0000 7f417e950ec0 20 remove_watcher() i=0 2019-12-09T17:09:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.129+0000 7f417e950ec0 2 removed watcher, disabling cache 2019-12-09T17:09:53.184 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.182+0000 7f417e950ec0 20 remove_watcher() i=1 2019-12-09T17:09:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.240+0000 7f417e950ec0 20 remove_watcher() i=2 2019-12-09T17:09:53.301 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.299+0000 7f417e950ec0 20 remove_watcher() i=3 2019-12-09T17:09:53.356 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.354+0000 7f417e950ec0 20 remove_watcher() i=4 2019-12-09T17:09:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.416+0000 7f417e950ec0 20 remove_watcher() i=5 2019-12-09T17:09:53.491 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.489+0000 7f417e950ec0 20 remove_watcher() i=6 2019-12-09T17:09:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2019-12-09T17:09:53.544+0000 7f417e950ec0 20 remove_watcher() i=7 2019-12-09T17:09:53.554 INFO:tasks.swift:Configuring testswift... 2019-12-09T17:09:53.554 INFO:teuthology.orchestra.run.smithi059:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-12-09T17:09:56.025 INFO:teuthology.orchestra.run.smithi059.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-12-09T17:09:57.436 INFO:teuthology.orchestra.run.smithi059.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T17:09:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T17:09:57.952 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting pip 2019-12-09T17:09:58.036 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T17:09:58.105 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: pip 2019-12-09T17:09:58.105 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: pip 9.0.1 2019-12-09T17:09:58.166 INFO:teuthology.orchestra.run.smithi059.stdout: Uninstalling pip-9.0.1: 2019-12-09T17:09:58.231 INFO:teuthology.orchestra.run.smithi059.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T17:09:58.974 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed pip-19.3.1 2019-12-09T17:09:59.249 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T17:09:59.912 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting setuptools==32.3.1 2019-12-09T17:09:59.916 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T17:09:59.963 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: setuptools 2019-12-09T17:09:59.964 INFO:teuthology.orchestra.run.smithi059.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T17:09:59.979 INFO:teuthology.orchestra.run.smithi059.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T17:09:59.988 INFO:teuthology.orchestra.run.smithi059.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T17:10:00.122 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed setuptools-32.3.1 2019-12-09T17:10:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T17:10:00.748 INFO:teuthology.orchestra.run.smithi059.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T17:10:00.809 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting nose>=1.0.0 2019-12-09T17:10:00.811 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T17:10:00.955 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting boto>=2.0b4 2019-12-09T17:10:00.964 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T17:10:01.119 INFO:teuthology.orchestra.run.smithi059.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T17:10:01.423 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting gevent>=0.14 2019-12-09T17:10:01.440 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T17:10:01.702 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting isodate>=0.4.4 2019-12-09T17:10:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-12-09T17:10:01.793 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting eventlet>=0.9.14 2019-12-09T17:10:01.847 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-12-09T17:10:02.046 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting netifaces>=0.5 2019-12-09T17:10:02.057 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T17:10:02.153 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting xattr<0.7.0,>=0.6 2019-12-09T17:10:02.164 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-12-09T17:10:02.642 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting simplejson>=2.1.3 2019-12-09T17:10:02.654 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-12-09T17:10:03.075 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting WebOb==1.0 2019-12-09T17:10:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-12-09T17:10:03.569 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T17:10:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T17:10:03.657 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting six 2019-12-09T17:10:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T17:10:03.725 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting monotonic>=1.4 2019-12-09T17:10:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-12-09T17:10:03.836 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting dnspython>=1.15.0 2019-12-09T17:10:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-12-09T17:10:04.002 INFO:teuthology.orchestra.run.smithi059.stdout:Collecting enum34; python_version < "3.4" 2019-12-09T17:10:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-12-09T17:10:04.050 INFO:teuthology.orchestra.run.smithi059.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-12-09T17:10:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for xattr (setup.py): started 2019-12-09T17:10:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-12-09T17:10:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24121 sha256=340cbd86c4e23494fc1757f8aba50f02c3b9fd82a0102786eda387a30042b98d 2019-12-09T17:10:04.605 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-12-09T17:10:04.612 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for simplejson (setup.py): started 2019-12-09T17:10:05.511 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-12-09T17:10:05.512 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113125 sha256=cab714b16a03ea35eeae11c842dcde41b9ac088885cf0bc24b7cad2cdaa6cc42 2019-12-09T17:10:05.513 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-12-09T17:10:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for WebOb (setup.py): started 2019-12-09T17:10:05.864 INFO:teuthology.orchestra.run.smithi059.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-12-09T17:10:05.864 INFO:teuthology.orchestra.run.smithi059.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=a5ee81eb726ba751e43042f3c1af43b46bb9c1c16c8b30e2dc640b2d3a61f996 2019-12-09T17:10:05.864 INFO:teuthology.orchestra.run.smithi059.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-12-09T17:10:05.873 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully built xattr simplejson WebOb 2019-12-09T17:10:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-12-09T17:10:07.394 INFO:teuthology.orchestra.run.smithi059.stdout:Successfully installed PyYAML-5.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-12-09T17:10:07.711 INFO:teuthology.orchestra.run.smithi059.stdout:running develop 2019-12-09T17:10:07.762 INFO:teuthology.orchestra.run.smithi059.stdout:running egg_info 2019-12-09T17:10:07.763 INFO:teuthology.orchestra.run.smithi059.stdout:creating swift.egg-info 2019-12-09T17:10:07.763 INFO:teuthology.orchestra.run.smithi059.stdout:writing swift.egg-info/PKG-INFO 2019-12-09T17:10:07.763 INFO:teuthology.orchestra.run.smithi059.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-12-09T17:10:07.763 INFO:teuthology.orchestra.run.smithi059.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-12-09T17:10:07.768 INFO:teuthology.orchestra.run.smithi059.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-12-09T17:10:07.768 INFO:teuthology.orchestra.run.smithi059.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T17:10:07.771 INFO:teuthology.orchestra.run.smithi059.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T17:10:07.771 INFO:teuthology.orchestra.run.smithi059.stdout:reading manifest template 'MANIFEST.in' 2019-12-09T17:10:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:warning: no files found matching 'ChangeLog' 2019-12-09T17:10:07.775 INFO:teuthology.orchestra.run.smithi059.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T17:10:07.776 INFO:teuthology.orchestra.run.smithi059.stdout:running build_ext 2019-12-09T17:10:07.777 INFO:teuthology.orchestra.run.smithi059.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-12-09T17:10:07.777 INFO:teuthology.orchestra.run.smithi059.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-12-09T17:10:07.794 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.794 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.797 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.806 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.806 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.806 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.807 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.807 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.808 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.809 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.810 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.810 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.810 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.811 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.811 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.811 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.811 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.812 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.814 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.815 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.815 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.818 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.820 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.828 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.829 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.830 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.831 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.834 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.835 INFO:teuthology.orchestra.run.smithi059.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:10:07.835 INFO:teuthology.orchestra.run.smithi059.stdout: 2019-12-09T17:10:07.836 INFO:teuthology.orchestra.run.smithi059.stdout:Installed /home/ubuntu/cephtest/swift 2019-12-09T17:10:07.836 INFO:teuthology.orchestra.run.smithi059.stdout:Processing dependencies for swift==1.4.4 2019-12-09T17:10:07.837 INFO:teuthology.orchestra.run.smithi059.stdout:Finished processing dependencies for swift==1.4.4 2019-12-09T17:10:07.849 INFO:teuthology.orchestra.run.smithi059:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-12-09T17:10:07.905 INFO:teuthology.orchestra.run.smithi059:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-12-09T17:10:19.694 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:27:16.315 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.701 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:27:16.322 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.709 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:27:16.331 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.716 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:27:16.337 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.728 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:27:16.328 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.735 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:27:16.335 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.835 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:27:16.456 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.836 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:27:16.457 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:19.860 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:27:16.460 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:41.696 INFO:teuthology.orchestra.run.smithi059.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-12-09T17:10:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-12-09T17:15:32.344 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:32:28.965 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.368 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:32:28.971 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.369 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:32:28.969 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.369 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:32:28.990 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.374 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:32:28.974 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.374 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:32:28.995 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.571 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:32:29.192 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.575 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:32:29.176 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:32.600 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:32:29.221 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.901 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:32:57.521 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.933 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:32:57.554 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.942 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:32:57.542 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.944 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:32:57.544 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.962 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:32:57.582 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.966 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:32:57.588 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.980 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:32:57.601 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:00.984 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:32:57.585 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:01.004 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:32:57.625 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:16:29.528 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-12-09T17:16:43.166 INFO:teuthology.orchestra.run.smithi059.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-12-09T17:16:48.225 INFO:teuthology.orchestra.run.smithi059.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-12-09T17:17:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-12-09T17:17:18.725 INFO:teuthology.orchestra.run.smithi059.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-12-09T17:18:12.778 INFO:teuthology.orchestra.run.smithi059.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-12-09T17:18:13.450 INFO:teuthology.orchestra.run.smithi059.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-12-09T17:20:44.532 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:37:41.153 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.548 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:37:41.166 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.560 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:37:41.182 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.561 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:37:41.161 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.567 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:37:41.188 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.576 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:37:41.176 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.764 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:37:41.385 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.769 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:37:41.390 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:44.802 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:37:41.403 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.636 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:41:19.247 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.653 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:41:19.275 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.669 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:41:19.290 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.694 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:41:19.315 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.714 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:41:19.314 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.722 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:41:19.322 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.807 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:41:19.429 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.808 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:41:19.430 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:24:22.844 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:41:19.445 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:45.000 INFO:teuthology.orchestra.run.smithi059.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-12-09T17:25:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-12-09T17:25:57.993 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:42:54.614 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.006 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:42:54.627 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.014 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:42:54.635 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.022 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:42:54.643 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.023 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:42:54.621 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.027 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:42:54.627 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.101 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:42:54.722 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.111 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:42:54.732 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:58.177 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:42:54.778 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:20.319 INFO:teuthology.orchestra.run.smithi059.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:26:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:31:10.272 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:48:06.891 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.292 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:48:06.895 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.293 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:48:06.904 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.293 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:48:06.906 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.300 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:48:06.901 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.309 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:48:06.910 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.491 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:48:07.112 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.527 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:48:07.128 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:10.533 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:48:07.154 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:31:38.749 INFO:teuthology.orchestra.run.smithi059.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:31:44.026 INFO:teuthology.orchestra.run.smithi059.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:31:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:32:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:32:44.591 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:49:41.212 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.598 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:49:41.220 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.655 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:49:41.277 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.657 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:49:41.278 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.712 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:49:41.304 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.712 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:49:41.307 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.730 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:49:41.352 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.767 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:49:41.389 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:44.802 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:49:41.403 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.248 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:49:42.849 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.250 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:49:42.851 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.272 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:49:42.872 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.392 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:49:43.013 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.394 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:49:43.014 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.413 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:49:43.030 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.416 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:49:43.037 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.417 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:49:43.038 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:32:46.430 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:49:43.051 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:13.018 INFO:teuthology.orchestra.run.smithi059.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:34:13.784 INFO:teuthology.orchestra.run.smithi059.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T17:36:21.502 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:53:18.124 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.510 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:53:18.132 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.523 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:53:18.124 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.529 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:53:18.130 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.534 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:53:18.155 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.537 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:53:18.159 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.665 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:53:18.266 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.675 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:53:18.297 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:21.685 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:53:18.306 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.467 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:58:03.088 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.494 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:58:03.115 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.511 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:58:03.131 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.531 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:58:03.152 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.552 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:58:03.152 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.556 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:58:03.157 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.677 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:58:03.298 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.688 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:58:03.309 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:06.791 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:58:03.391 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.213 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:58:29.833 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.220 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:58:29.842 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.227 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:58:29.848 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.229 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:58:29.850 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.238 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:58:29.839 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.245 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:58:29.846 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.273 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:07:58:29.894 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.282 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:07:58:29.897 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:33.304 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:07:58:29.898 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainer) ... ok 2019-12-09T17:41:51.279 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T17:41:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T17:41:56.654 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T17:41:57.229 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerInfo (test.functional.tests.TestContainer) ... ok 2019-12-09T17:41:57.871 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T17:42:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:testContainerNameLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T17:42:17.912 INFO:teuthology.orchestra.run.smithi059.stderr:testCreate (test.functional.tests.TestContainer) ... ok 2019-12-09T17:42:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:testDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T17:42:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T17:42:40.905 INFO:teuthology.orchestra.run.smithi059.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainer) ... ok 2019-12-09T17:42:41.612 INFO:teuthology.orchestra.run.smithi059.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T17:44:32.324 INFO:teuthology.orchestra.run.smithi059.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok 2019-12-09T17:44:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:testFileOrder (test.functional.tests.TestContainer) ... ok 2019-12-09T17:44:51.299 INFO:teuthology.orchestra.run.smithi059.stderr:testFileThenContainerDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T17:44:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:testLastFileMarker (test.functional.tests.TestContainer) ... ok 2019-12-09T17:45:33.475 INFO:teuthology.orchestra.run.smithi059.stderr:testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T17:46:45.387 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:03:42.008 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.393 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:03:42.013 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.412 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:03:42.033 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.413 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:03:42.013 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.417 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:03:42.038 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.425 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:03:42.024 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.585 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:03:42.206 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.634 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:03:42.255 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:45.649 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:03:42.249 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:47:00.568 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:00.558+0000 1694f700 -1 osd.6 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:18.713325+0000 front 2019-12-09T17:46:18.701520+0000 (oldest deadline 2019-12-09T17:47:00.075904+0000) 2019-12-09T17:47:01.040 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:47:01.022+0000 1694f700 -1 osd.1 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:20.627901+0000 front 2019-12-09T17:46:18.808421+0000 (oldest deadline 2019-12-09T17:47:00.450098+0000) 2019-12-09T17:47:01.040 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:01.020+0000 1694f700 -1 osd.4 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:16.419982+0000 front 2019-12-09T17:46:16.174961+0000 (oldest deadline 2019-12-09T17:47:00.745060+0000) 2019-12-09T17:47:01.615 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:01.610+0000 1694f700 -1 osd.6 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:18.713325+0000 front 2019-12-09T17:46:18.701520+0000 (oldest deadline 2019-12-09T17:47:00.075904+0000) 2019-12-09T17:47:01.769 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:47:01.750+0000 1694f700 -1 osd.5 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:14.958078+0000 front 2019-12-09T17:46:16.495798+0000 (oldest deadline 2019-12-09T17:47:00.792142+0000) 2019-12-09T17:47:01.999 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:01.994+0000 1694f700 -1 osd.4 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:16.419982+0000 front 2019-12-09T17:46:16.174961+0000 (oldest deadline 2019-12-09T17:47:00.745060+0000) 2019-12-09T17:47:02.002 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:47:01.999+0000 1694f700 -1 osd.1 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:20.627901+0000 front 2019-12-09T17:46:18.808421+0000 (oldest deadline 2019-12-09T17:47:00.450098+0000) 2019-12-09T17:47:02.657 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:02.652+0000 1694f700 -1 osd.6 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:18.713325+0000 front 2019-12-09T17:46:18.701520+0000 (oldest deadline 2019-12-09T17:47:00.075904+0000) 2019-12-09T17:47:02.670 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:47:02.658+0000 1694f700 -1 osd.2 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:18.913090+0000 front 2019-12-09T17:46:18.968874+0000 (oldest deadline 2019-12-09T17:47:01.708017+0000) 2019-12-09T17:47:02.792 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:47:02.784+0000 1694f700 -1 osd.0 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:19.676800+0000 front 2019-12-09T17:46:18.683072+0000 (oldest deadline 2019-12-09T17:47:02.754984+0000) 2019-12-09T17:47:02.815 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:47:02.811+0000 1694f700 -1 osd.5 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:14.958078+0000 front 2019-12-09T17:46:16.495798+0000 (oldest deadline 2019-12-09T17:47:00.792142+0000) 2019-12-09T17:47:02.990 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:02.963+0000 1694f700 -1 osd.4 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:16.419982+0000 front 2019-12-09T17:46:16.174961+0000 (oldest deadline 2019-12-09T17:47:00.745060+0000) 2019-12-09T17:47:03.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:47:03.022+0000 1694f700 -1 osd.1 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:20.627901+0000 front 2019-12-09T17:46:18.808421+0000 (oldest deadline 2019-12-09T17:47:00.450098+0000) 2019-12-09T17:47:03.658 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:47:03.654+0000 1694f700 -1 osd.2 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:18.913090+0000 front 2019-12-09T17:46:18.968874+0000 (oldest deadline 2019-12-09T17:47:01.708017+0000) 2019-12-09T17:47:03.711 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:03.697+0000 1694f700 -1 osd.6 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:18.713325+0000 front 2019-12-09T17:46:18.701520+0000 (oldest deadline 2019-12-09T17:47:00.075904+0000) 2019-12-09T17:47:03.809 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:47:03.804+0000 1694f700 -1 osd.5 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:14.958078+0000 front 2019-12-09T17:46:16.495798+0000 (oldest deadline 2019-12-09T17:47:00.792142+0000) 2019-12-09T17:47:03.831 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:47:03.825+0000 1694f700 -1 osd.0 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:19.676800+0000 front 2019-12-09T17:46:18.683072+0000 (oldest deadline 2019-12-09T17:47:02.754984+0000) 2019-12-09T17:47:03.956 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:03.950+0000 1694f700 -1 osd.4 139 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:46:16.419982+0000 front 2019-12-09T17:46:16.174961+0000 (oldest deadline 2019-12-09T17:47:00.745060+0000) 2019-12-09T17:47:37.170 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.122+0000 38a7d700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.242 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.149+0000 33a73700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.268 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.168+0000 38a7d700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.281 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.180+0000 33a73700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.471 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.265+0000 34274700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.383+0000 34274700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.716 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.475+0000 124b0700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.756 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:47:37.563+0000 34274700 -1 osd.0 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.759 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:47:37.571+0000 3827c700 -1 osd.0 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.770 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:47:37.581+0000 34274700 -1 osd.0 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.822 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:37.750+0000 33a73700 -1 osd.6 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.834 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:47:37.721+0000 36a79700 -1 osd.4 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.881 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:37.803+0000 34274700 -1 osd.6 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:37.921 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:37.805+0000 34274700 -1 osd.6 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:38.056 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:47:37.908+0000 3aa81700 -1 osd.0 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:38.522 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:38.242+0000 34274700 -1 osd.6 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:47:38.561 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:47:38.489+0000 34a75700 -1 osd.6 144 build_incremental_map_msg missing incremental map 144 2019-12-09T17:48:07.252 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:48:07.249+0000 1694f700 -1 osd.3 146 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:117393 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e144) 2019-12-09T17:48:07.409 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:48:07.406+0000 1694f700 -1 osd.0 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.666759+0000 front 2019-12-09T17:47:24.933621+0000 (oldest deadline 2019-12-09T17:48:06.627993+0000) 2019-12-09T17:48:07.856 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:48:07.850+0000 1694f700 -1 osd.4 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.305545+0000 front 2019-12-09T17:47:26.026885+0000 (oldest deadline 2019-12-09T17:48:07.541341+0000) 2019-12-09T17:48:08.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:48:08.250+0000 1694f700 -1 osd.3 146 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:117393 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e144) 2019-12-09T17:48:08.348 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:48:08.346+0000 1694f700 -1 osd.2 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.304597+0000 front 2019-12-09T17:47:26.305537+0000 (oldest deadline 2019-12-09T17:48:08.161388+0000) 2019-12-09T17:48:08.427 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:48:08.423+0000 1694f700 -1 osd.0 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.666759+0000 front 2019-12-09T17:47:24.933621+0000 (oldest deadline 2019-12-09T17:48:06.627993+0000) 2019-12-09T17:48:08.875 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:48:08.871+0000 1694f700 -1 osd.4 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.305545+0000 front 2019-12-09T17:47:26.026885+0000 (oldest deadline 2019-12-09T17:48:07.541341+0000) 2019-12-09T17:48:09.249 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:48:09.245+0000 1694f700 -1 osd.5 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.306690+0000 front 2019-12-09T17:47:26.325835+0000 (oldest deadline 2019-12-09T17:48:08.793473+0000) 2019-12-09T17:48:09.263 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:48:09.261+0000 1694f700 -1 osd.3 146 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:117393 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e144) 2019-12-09T17:48:09.353 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:48:09.336+0000 1694f700 -1 osd.2 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.304597+0000 front 2019-12-09T17:47:26.305537+0000 (oldest deadline 2019-12-09T17:48:08.161388+0000) 2019-12-09T17:48:09.461 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:48:09.459+0000 1694f700 -1 osd.0 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.666759+0000 front 2019-12-09T17:47:24.933621+0000 (oldest deadline 2019-12-09T17:48:06.627993+0000) 2019-12-09T17:48:09.618 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:48:09.607+0000 1694f700 -1 osd.7 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:25.636184+0000 front 2019-12-09T17:47:25.705126+0000 (oldest deadline 2019-12-09T17:48:08.822824+0000) 2019-12-09T17:48:09.870 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:48:09.861+0000 1694f700 -1 osd.4 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.305545+0000 front 2019-12-09T17:47:26.026885+0000 (oldest deadline 2019-12-09T17:48:07.541341+0000) 2019-12-09T17:48:10.219 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:48:10.215+0000 1694f700 -1 osd.5 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.306690+0000 front 2019-12-09T17:47:26.325835+0000 (oldest deadline 2019-12-09T17:48:08.793473+0000) 2019-12-09T17:48:10.323 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:48:10.288+0000 1694f700 -1 osd.3 147 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:117393 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e144) 2019-12-09T17:48:10.438 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:48:10.435+0000 1694f700 -1 osd.0 146 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:47:26.666759+0000 front 2019-12-09T17:47:24.933621+0000 (oldest deadline 2019-12-09T17:48:06.627993+0000) 2019-12-09T17:49:28.176 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:06:24.797 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.224 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:06:24.845 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.225 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:06:24.846 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.234 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:06:24.834 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.241 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:06:24.861 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.242 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:06:24.841 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.390 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:06:24.990 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.418 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:06:25.039 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:28.495 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:06:25.116 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:29.629 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:29.626+0000 34274700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:29.636 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:29.628+0000 36a79700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:29.774 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:29.764+0000 39a7f700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:29.835 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:29.783+0000 36a79700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:29.933 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:29.833+0000 39a7f700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:30.077 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:29.927+0000 36278700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:30.367 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:30.279+0000 39a7f700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:30.416 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:30.302+0000 124b0700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:30.417 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:30.303+0000 3b282700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:49:30.534 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:49:30.369+0000 3827c700 -1 osd.6 151 build_incremental_map_msg missing incremental map 151 2019-12-09T17:50:31.262 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:50:31.256+0000 1694f700 -1 osd.5 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.231202+0000 front 2019-12-09T17:49:46.582061+0000 (oldest deadline 2019-12-09T17:50:30.531705+0000) 2019-12-09T17:50:31.534 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:50:31.529+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.068486+0000 front 2019-12-09T17:49:47.669553+0000 (oldest deadline 2019-12-09T17:50:31.141690+0000) 2019-12-09T17:50:32.102 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:50:32.096+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.426204+0000 front 2019-12-09T17:49:46.663147+0000 (oldest deadline 2019-12-09T17:50:31.780429+0000) 2019-12-09T17:50:32.180 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:50:32.176+0000 1694f700 -1 osd.2 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.127804+0000 front 2019-12-09T17:49:48.128779+0000 (oldest deadline 2019-12-09T17:50:31.286626+0000) 2019-12-09T17:50:32.309 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:50:32.303+0000 1694f700 -1 osd.5 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.231202+0000 front 2019-12-09T17:49:46.582061+0000 (oldest deadline 2019-12-09T17:50:30.531705+0000) 2019-12-09T17:50:32.582 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:50:32.568+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.068486+0000 front 2019-12-09T17:49:47.669553+0000 (oldest deadline 2019-12-09T17:50:31.141690+0000) 2019-12-09T17:50:32.703 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:50:32.696+0000 1694f700 -1 osd.6 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.521437+0000 front 2019-12-09T17:49:49.410328+0000 (oldest deadline 2019-12-09T17:50:32.464599+0000) 2019-12-09T17:50:33.141 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:50:33.135+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.426204+0000 front 2019-12-09T17:49:46.663147+0000 (oldest deadline 2019-12-09T17:50:31.780429+0000) 2019-12-09T17:50:33.192 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:50:33.187+0000 1694f700 -1 osd.2 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.127804+0000 front 2019-12-09T17:49:48.128779+0000 (oldest deadline 2019-12-09T17:50:31.286626+0000) 2019-12-09T17:50:33.359 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:50:33.354+0000 1694f700 -1 osd.5 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.231202+0000 front 2019-12-09T17:49:46.582061+0000 (oldest deadline 2019-12-09T17:50:30.531705+0000) 2019-12-09T17:50:33.633 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:50:33.629+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.068486+0000 front 2019-12-09T17:49:47.669553+0000 (oldest deadline 2019-12-09T17:50:31.141690+0000) 2019-12-09T17:50:33.693 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:50:33.687+0000 1694f700 -1 osd.6 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.521437+0000 front 2019-12-09T17:49:49.410328+0000 (oldest deadline 2019-12-09T17:50:32.464599+0000) 2019-12-09T17:50:34.103 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:50:34.098+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.426204+0000 front 2019-12-09T17:49:46.663147+0000 (oldest deadline 2019-12-09T17:50:31.780429+0000) 2019-12-09T17:50:34.154 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:50:34.149+0000 1694f700 -1 osd.2 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.127804+0000 front 2019-12-09T17:49:48.128779+0000 (oldest deadline 2019-12-09T17:50:31.286626+0000) 2019-12-09T17:50:34.336 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:50:34.331+0000 1694f700 -1 osd.5 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.231202+0000 front 2019-12-09T17:49:46.582061+0000 (oldest deadline 2019-12-09T17:50:30.531705+0000) 2019-12-09T17:50:34.455 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:50:34.451+0000 1694f700 -1 osd.4 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:49.292908+0000 front 2019-12-09T17:49:49.316869+0000 (oldest deadline 2019-12-09T17:50:34.301871+0000) 2019-12-09T17:50:34.633 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:50:34.625+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.068486+0000 front 2019-12-09T17:49:47.669553+0000 (oldest deadline 2019-12-09T17:50:31.141690+0000) 2019-12-09T17:50:34.692 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:50:34.687+0000 1694f700 -1 osd.6 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.521437+0000 front 2019-12-09T17:49:49.410328+0000 (oldest deadline 2019-12-09T17:50:32.464599+0000) 2019-12-09T17:50:35.078 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:50:35.072+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.426204+0000 front 2019-12-09T17:49:46.663147+0000 (oldest deadline 2019-12-09T17:50:31.780429+0000) 2019-12-09T17:50:35.164 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:50:35.159+0000 1694f700 -1 osd.2 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:48.127804+0000 front 2019-12-09T17:49:48.128779+0000 (oldest deadline 2019-12-09T17:50:31.286626+0000) 2019-12-09T17:50:35.381 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:50:35.376+0000 1694f700 -1 osd.5 152 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:49:47.231202+0000 front 2019-12-09T17:49:46.582061+0000 (oldest deadline 2019-12-09T17:50:30.531705+0000) 2019-12-09T17:50:49.699 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:07:46.319 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.704 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:07:46.324 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.718 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:07:46.338 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.720 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:07:46.320 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.721 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:07:46.341 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.727 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:07:46.327 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.820 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:07:46.440 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.821 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:07:46.442 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:50:49.868 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:07:46.467 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:31.979 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:31.975+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:32.798 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:32.791+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:32.942 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:32.938+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:33.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:33.511+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:33.754 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:33.750+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:33.968 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:33.956+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:34.200 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:34.195+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:34.508 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:34.504+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:34.724 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:34.719+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:34.986 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:34.981+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:35.043 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:35.039+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:35.228 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:35.223+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:35.375 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:35.370+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:35.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:35.476+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:35.714 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:35.710+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:35.954 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:35.950+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:36.002 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:35.998+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:36.218 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:36.213+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:36.416 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:36.411+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:36.446 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:36.442+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:36.675 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:36.671+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:36.942 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:36.938+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:37.032 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:37.027+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:37.227 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:37.220+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:37.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:37.418+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:37.472 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:37.468+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:37.712 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:37.708+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:37.907 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:37.903+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:38.030 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:38.022+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:38.240 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:38.236+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:38.440 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:38.436+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:38.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:38.489+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:38.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:38.683+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:38.874 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:38.871+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:39.027 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:39.022+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:39.247 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:39.242+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:39.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:39.475+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:39.492 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:39.486+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:39.671 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:39.666+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:39.905 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:39.894+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:40.079 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:40.074+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:40.224 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:40.220+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:40.503 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:40.499+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:40.521 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:40.516+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:40.699 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:40.695+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:40.875 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:40.871+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:41.060 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:41.056+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:41.244 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:41.239+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:41.532 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:41.528+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:41.555 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:41.549+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:41.689 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:41.685+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:41.892 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:41.885+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:42.041 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:42.034+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:42.255 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:42.250+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:42.532 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:42.528+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:42.591 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:42.587+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:42.720 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:42.716+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:51:42.884 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:51:42.879+0000 1694f700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:51.398976+0000 front 2019-12-09T17:50:51.450542+0000 (oldest deadline 2019-12-09T17:51:31.917394+0000) 2019-12-09T17:51:43.068 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:51:43.062+0000 1694f700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.042244+0000 front 2019-12-09T17:50:50.788383+0000 (oldest deadline 2019-12-09T17:51:34.657971+0000) 2019-12-09T17:51:43.247 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:51:43.241+0000 1694f700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:50.806680+0000 front 2019-12-09T17:50:52.038967+0000 (oldest deadline 2019-12-09T17:51:33.348291+0000) 2019-12-09T17:51:43.578 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:51:43.573+0000 1694f700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.046182+0000 front 2019-12-09T17:50:50.804872+0000 (oldest deadline 2019-12-09T17:51:34.795986+0000) 2019-12-09T17:51:43.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:51:43.596+0000 1694f700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:53.205954+0000 front 2019-12-09T17:50:53.183282+0000 (oldest deadline 2019-12-09T17:51:33.398831+0000) 2019-12-09T17:51:43.748 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:51:43.743+0000 1694f700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:50:52.583032+0000 front 2019-12-09T17:50:52.090829+0000 (oldest deadline 2019-12-09T17:51:31.949194+0000) 2019-12-09T17:53:56.636 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:53:56.631+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:53:57.676 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:53:57.672+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:53:58.695 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:53:58.691+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:53:59.726 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:53:59.722+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:00.717 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:00.712+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:01.716 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:01.712+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:02.747 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:02.743+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:03.801 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:03.797+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:04.798 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:04.793+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:05.782 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:05.778+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:06.797 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:06.793+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:07.819 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:07.815+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:08.818 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:08.814+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:09.815 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:09.810+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:10.785 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:10.781+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:11.807 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:11.802+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:12.773 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:12.769+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:13.507 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:13.502+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:13.805 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:13.799+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:13.806 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:13.802+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:14.365 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:14.358+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:14.501 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:14.488+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:14.525 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:14.519+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:14.701 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:14.697+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:14.797 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:14.792+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:14.800 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:14.796+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:15.123 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:15.118+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:15.358 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:15.353+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:15.488 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:15.483+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:15.510 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:15.504+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:15.715 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:15.710+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:15.749 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:15.743+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:15.781 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:15.776+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:15.785 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:15.780+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:16.112 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:16.107+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:16.353 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:16.348+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:16.468 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:16.464+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:16.493 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:16.488+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:16.747 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:16.743+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:16.759 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:16.755+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:16.773 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:16.768+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:16.776 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:16.773+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:17.072 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:17.067+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:17.325 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:17.320+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:17.485 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:17.479+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:17.486 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:17.482+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:17.747 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:17.742+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:17.762 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:17.756+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:17.817 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:17.813+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:17.820 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:17.816+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:18.047 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:18.042+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:18.371 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:18.366+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:18.445 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:18.439+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:18.478 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:18.472+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:18.725 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:18.720+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:18.729 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:18.721+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:18.785 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:18.776+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:18.788 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:18.779+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:19.070 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:19.065+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:19.333 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:19.327+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:19.447 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:19.443+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:19.490 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:19.483+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:19.708 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:19.701+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:19.744 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:19.737+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:19.776 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:19.771+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:19.777 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:19.772+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:20.100 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:20.095+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:20.384 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:20.379+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:20.423 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:20.417+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:20.505 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:20.502+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:20.663 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:20.658+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:20.792 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:20.787+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:20.813 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:20.808+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:20.814 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:20.810+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:21.140 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:21.136+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:21.397 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:21.393+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:21.436 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:21.433+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:21.490 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:21.485+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:21.692 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:21.686+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:21.773 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:21.767+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:21.826 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:21.822+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:21.827 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:21.824+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:22.173 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:22.168+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:22.451 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:22.447+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:22.477 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:22.473+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:22.483 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:22.478+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:22.742 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:22.735+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:22.805 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:22.800+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:22.820 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:22.815+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:22.821 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:22.818+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:23.174 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:23.169+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:23.439 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:23.435+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:23.497 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:23.492+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:23.497 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:23.492+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:23.791 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:23.786+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:23.812 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:23.808+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:23.815 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:23.813+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:23.825 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:23.819+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:24.229 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:24.224+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:24.460 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:24.455+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:24.471 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:24.466+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:24.541 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:24.536+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:24.761 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:24.756+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:24.840 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:24.836+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:24.841 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:24.835+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:24.844 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:24.841+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:25.253 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:25.248+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:25.490 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:25.484+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:25.507 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:25.503+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:25.590 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:25.586+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:25.765 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:25.761+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:25.799 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:25.796+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:25.803 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:25.800+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:25.874 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:25.868+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:26.304 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:26.299+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:26.452 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:26.447+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:26.467 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:26.463+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:26.585 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:26.581+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:26.746 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:26.741+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:26.806 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:26.801+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:26.808 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:26.804+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:26.837 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:26.831+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:27.321 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:27.318+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:27.483 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:27.478+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:27.507 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:27.502+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:27.605 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:27.592+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:27.732 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:27.726+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:27.856 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:27.851+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:27.856 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:27.853+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:27.884 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:27.879+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:28.371 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:54:28.367+0000 1694f700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.404230+0000 front 2019-12-09T17:53:29.935462+0000 (oldest deadline 2019-12-09T17:54:14.666930+0000) 2019-12-09T17:54:28.463 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:54:28.457+0000 1694f700 -1 osd.1 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.855476+0000 front 2019-12-09T17:53:31.840313+0000 (oldest deadline 2019-12-09T17:54:12.507166+0000) 2019-12-09T17:54:28.511 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:28.507+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:28.615 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:54:28.612+0000 1694f700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.967985+0000 front 2019-12-09T17:53:28.778195+0000 (oldest deadline 2019-12-09T17:54:13.396194+0000) 2019-12-09T17:54:28.761 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:54:28.756+0000 1694f700 -1 osd.0 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:29.857334+0000 front 2019-12-09T17:53:30.066614+0000 (oldest deadline 2019-12-09T17:54:13.927576+0000) 2019-12-09T17:54:28.861 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:28.858+0000 1694f700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:28.444547+0000 front 2019-12-09T17:53:28.460125+0000 (oldest deadline 2019-12-09T17:54:13.697876+0000) 2019-12-09T17:54:28.863 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:28.861+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:28.935 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:54:28.931+0000 1694f700 -1 osd.2 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:32.074982+0000 front 2019-12-09T17:53:32.407952+0000 (oldest deadline 2019-12-09T17:54:14.941391+0000) 2019-12-09T17:54:29.502 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:54:29.498+0000 1694f700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:53:31.867806+0000 front 2019-12-09T17:53:30.499978+0000 (oldest deadline 2019-12-09T17:54:13.734932+0000) 2019-12-09T17:54:30.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:30.241+0000 1694f700 -1 osd.4 164 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:31.289 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:54:31.268+0000 1694f700 -1 osd.4 165 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118206 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T17:54:57.796 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:11:54.416 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:57.803 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:11:54.423 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:57.811 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:11:54.430 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:57.815 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:11:54.435 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:57.830 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:11:54.430 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:57.846 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:11:54.446 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:58.095 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:11:54.715 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:58.115 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:11:54.714 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:58.170 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:11:54.790 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:35.176 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:55:35.097+0000 3827c700 -1 osd.0 167 build_incremental_map_msg missing incremental map 167 2019-12-09T17:55:35.391 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:55:35.142+0000 35a77700 -1 osd.0 167 build_incremental_map_msg missing incremental map 167 2019-12-09T17:55:35.506 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:55:35.163+0000 35a77700 -1 osd.0 167 build_incremental_map_msg missing incremental map 167 2019-12-09T17:55:35.508 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:55:35.169+0000 35276700 -1 osd.0 167 build_incremental_map_msg missing incremental map 167 2019-12-09T17:55:35.536 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:55:35.273+0000 34a75700 -1 osd.0 167 build_incremental_map_msg missing incremental map 167 2019-12-09T17:55:35.551 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:55:35.334+0000 3a280700 -1 osd.0 167 build_incremental_map_msg missing incremental map 167 2019-12-09T17:55:43.624 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:55:43.614+0000 3727a700 -1 osd.4 169 build_incremental_map_msg missing incremental map 144 2019-12-09T17:55:49.713 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:55:49.705+0000 38a7d700 -1 osd.6 169 build_incremental_map_msg missing incremental map 144 2019-12-09T17:55:50.511 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:55:50.497+0000 38a7d700 -1 osd.6 169 build_incremental_map_msg missing incremental map 151 2019-12-09T17:56:02.876 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:02.872+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:03.203 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:03.192+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:03.879 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:03.873+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:04.262 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:04.258+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:04.927 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:04.922+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:05.019 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:05.015+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:05.234 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:05.229+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:05.382 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:05.377+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:05.937 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:05.932+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:06.054 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:06.043+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:06.284 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:06.270+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:06.398 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:06.393+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:06.770 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:06.765+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:06.818 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:06.814+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:06.914 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:06.909+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:07.047 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:07.040+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:07.307 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:07.300+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:07.308 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:07.304+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:07.427 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:07.422+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:07.794 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:07.791+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:07.816 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:07.811+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:07.965 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:07.960+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:08.044 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:08.037+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:08.302 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:08.299+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:08.304 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:08.301+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:08.475 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:08.470+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:08.821 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:08.817+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:08.850 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:08.847+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:08.991 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:08.987+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:09.062 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:09.059+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:09.304 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:09.300+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:09.305 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:09.302+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:09.500 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:09.495+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:09.852 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:09.846+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:09.868 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:09.864+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:10.033 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:10.028+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:10.037 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:10.033+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:10.355 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:10.351+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:10.357 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:10.354+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:10.527 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:10.521+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:10.740 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:10.735+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:10.847 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:10.844+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:10.858 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:10.855+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:11.006 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:11.002+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:11.052 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:11.048+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:11.325 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:11.318+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:11.326 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:11.320+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:11.495 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:11.490+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:11.718 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:11.714+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:11.867 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:11.864+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:11.878 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:11.874+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:11.980 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:11.976+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:12.083 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:12.079+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:12.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:12.292+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:12.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:12.293+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:12.469 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:12.464+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:12.749 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:12.744+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:12.893 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:12.888+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:12.899 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:12.895+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:12.965 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:12.960+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:13.101 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:13.094+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:13.288 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:13.283+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:13.292 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:13.285+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:13.520 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:13.516+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:13.710 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:13.706+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:13.926 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:13.922+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:13.930 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:13.927+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:13.934 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:13.930+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:14.109 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:14.105+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:14.285 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:14.280+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:14.286 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:14.282+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:14.489 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:14.485+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:14.704 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:14.699+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:14.947 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:14.942+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:14.973 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:14.969+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:14.979 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:14.976+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:15.141 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:15.137+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:15.250 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:15.246+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:15.255 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:15.251+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:15.536 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:15.531+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:15.721 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:15.715+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:15.942 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:15.939+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:15.965 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:15.961+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:16.029 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:16.025+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:16.145 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:16.141+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:16.277 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:16.274+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:16.282 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:16.278+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:16.544 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:16.540+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:16.714 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:16.706+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:16.939 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:16.934+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:17.009 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:17.003+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:17.009 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:17.006+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:17.122 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:17.118+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:17.285 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:17.281+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:17.288 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:17.285+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:17.542 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:17.537+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:17.751 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:17.746+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:17.909 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:17.906+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:18.025 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:18.018+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:18.038 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:18.033+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:18.126 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:18.122+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:18.316 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:18.310+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:18.317 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:18.312+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:18.519 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:18.515+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:18.772 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:18.763+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:18.974 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:18.967+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:19.000 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:18.994+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:19.075 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:19.071+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:19.103 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:19.098+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:19.341 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:19.335+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:19.342 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:19.337+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:19.504 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:19.498+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:19.731 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:19.727+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:19.953 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:19.947+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:20.018 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:20.014+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:20.091 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:20.085+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:20.117 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:20.113+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:20.309 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:20.302+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:20.310 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:20.304+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:20.521 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:20.517+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:20.778 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:20.773+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:20.917 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:20.912+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:21.062 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:21.057+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:21.089 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:21.083+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:21.123 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:21.117+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:21.348 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:21.342+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:21.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:21.344+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:21.498 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:21.493+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:21.770 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:21.766+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:21.893 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:21.888+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:22.092 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:22.087+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:22.102 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:22.097+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:22.137 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:22.132+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:22.372 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:22.366+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:22.373 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:22.368+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:22.502 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:22.497+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:22.823 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:22.815+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:22.913 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:22.908+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:23.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:23.113+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:23.129 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:23.123+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:23.137 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:23.132+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:23.425 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:23.419+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:23.426 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:23.421+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:23.517 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:23.512+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:23.801 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:23.797+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:23.964 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:23.959+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:24.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:24.113+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:24.159 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:24.155+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:24.170 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:24.165+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:24.449 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:24.444+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:24.450 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:24.445+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:24.488 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:24.483+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:24.767 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:24.762+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:24.935 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:24.931+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:25.100 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:25.095+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:25.136 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:25.129+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:25.199 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:25.195+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:25.420 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:25.414+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:25.421 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:25.416+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:25.481 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:25.476+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:25.762 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:25.758+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:25.929 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:25.923+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:26.091 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:26.086+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:26.166 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:26.161+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:26.247 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:26.242+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:26.448 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:26.441+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:26.450 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:26.443+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:26.464 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:26.460+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:26.758 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:26.754+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:26.974 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:26.969+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:27.146 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:27.140+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:27.214 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:27.209+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:27.278 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:27.273+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:27.416 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:27.410+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:27.416 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:27.412+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:27.466 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:27.461+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:27.728 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:27.724+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:27.961 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:27.956+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:28.153 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:28.148+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:28.171 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:28.166+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:28.275 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:28.270+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:28.403 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:28.398+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:28.404 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:28.400+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:28.485 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:28.470+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:28.732 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:28.727+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:28.943 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:28.938+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:29.131 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:29.126+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:29.137 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:56:29.132+0000 1694f700 -1 osd.2 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:26.324393+0000 front 2019-12-09T17:55:26.630627+0000 (oldest deadline 2019-12-09T17:56:06.378025+0000) 2019-12-09T17:56:29.315 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:29.310+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:29.435 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:29.428+0000 1694f700 -1 osd.3 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:22.964887+0000 front 2019-12-09T17:55:26.162800+0000 (oldest deadline 2019-12-09T17:56:02.747599+0000) 2019-12-09T17:56:29.435 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:29.430+0000 1694f700 -1 osd.3 169 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:56:29.531 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:29.526+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:29.746 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:56:29.741+0000 1694f700 -1 osd.4 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.466030+0000 front 2019-12-09T17:55:25.406572+0000 (oldest deadline 2019-12-09T17:56:10.334005+0000) 2019-12-09T17:56:29.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:56:29.939+0000 1694f700 -1 osd.1 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:25.091230+0000 front 2019-12-09T17:55:25.975512+0000 (oldest deadline 2019-12-09T17:56:06.753074+0000) 2019-12-09T17:56:30.117 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:56:30.099+0000 1694f700 -1 osd.0 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:18.874166+0000 front 2019-12-09T17:55:18.965380+0000 (oldest deadline 2019-12-09T17:56:04.116963+0000) 2019-12-09T17:56:30.221 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:13:26.841 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.231 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:13:26.848 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.275 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:13:26.874 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.295 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:13:26.895 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.302 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:56:30.298+0000 1694f700 -1 osd.5 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:19.396469+0000 front 2019-12-09T17:55:25.283660+0000 (oldest deadline 2019-12-09T17:56:02.684741+0000) 2019-12-09T17:56:30.312 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:13:26.932 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.314 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:13:26.934 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.415 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:13:27.015 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.485 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:13:27.104 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:30.488 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:56:30.484+0000 1694f700 -1 osd.7 169 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:55:33.175400+0000 front 2019-12-09T17:55:25.976368+0000 (oldest deadline 2019-12-09T17:56:05.208418+0000) 2019-12-09T17:56:30.535 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:13:27.155 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:31.004 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:56:30.751+0000 1694f700 -1 osd.3 170 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e167) 2019-12-09T17:57:50.459 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:14:47.078 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.488 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:14:47.108 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.492 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:14:47.112 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.494 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:14:47.114 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.550 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:14:47.149 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.553 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:14:47.153 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.601 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:14:47.221 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.607 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:14:47.227 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:50.689 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:14:47.288 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:57.667 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:57:57.661+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:57:58.630 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:57:58.624+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:57:58.634 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:57:58.629+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:57:59.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:57:59.361+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:57:59.603 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:57:59.598+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:57:59.633 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:57:59.628+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:00.077 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:00.071+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:00.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:00.367+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:00.397 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:00.392+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:00.543 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:00.538+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:00.625 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:00.619+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:00.628 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:00.622+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:01.129 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:01.125+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:01.353 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:01.348+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:01.437 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:01.432+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:01.513 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:01.506+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:01.645 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:01.639+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:01.676 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:01.671+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:02.128 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:02.123+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:02.332 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:02.326+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:02.470 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:02.463+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:02.509 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:02.503+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:02.656 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:02.652+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:02.666 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:02.660+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:03.123 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:03.115+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:03.199 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:03.191+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:03.323 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:03.317+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:03.438 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:03.431+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:03.491 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:03.487+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:03.617 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:03.612+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:03.720 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:03.715+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:04.168 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:04.163+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:04.215 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:04.210+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:04.326 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:04.321+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:04.476 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:04.471+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:04.498 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:04.493+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:04.597 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:04.591+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:04.696 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:04.691+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:05.203 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:05.199+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:05.250 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:05.246+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:05.348 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:05.342+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:05.431 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:05.425+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:05.534 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:05.529+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:05.568 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:05.563+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:05.697 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:05.693+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:06.253 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:06.249+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:06.305 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:06.300+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:06.384 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:06.380+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:06.430 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:06.425+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:06.533 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:06.528+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:06.541 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:06.534+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:06.699 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:06.694+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:07.265 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:07.261+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:07.280 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:07.276+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:07.346 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:07.341+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:07.409 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:07.404+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:07.534 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:07.529+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:07.561 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:07.557+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:07.724 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:07.719+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:08.253 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:08.248+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:08.256 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:08.252+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:08.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:08.345+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:08.445 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:08.438+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:08.544 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:08.539+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:08.564 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:08.559+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:08.710 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:08.706+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:09.215 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:09.210+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:09.302 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:09.298+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:09.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:09.390+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:09.453 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:09.447+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:09.525 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:09.521+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:09.544 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:09.538+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:09.700 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:09.695+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:10.181 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:10.177+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:10.273 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:10.267+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:10.370 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:10.365+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:10.420 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:10.414+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:10.508 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:10.504+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:10.528 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:10.523+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:10.747 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:10.743+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:11.171 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:11.166+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:11.247 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:11.235+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:11.385 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:11.380+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:11.452 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:11.447+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:11.465 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:11.461+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:11.546 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:11.540+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:11.704 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:11.699+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:12.165 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:12.161+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:12.203 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:12.199+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:12.426 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:12.420+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:12.441 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:12.435+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:12.505 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:12.499+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:12.595 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:12.588+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:12.731 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:12.726+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:13.171 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:13.165+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:13.210 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:13.205+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:13.432 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:13.427+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:13.466 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:13.460+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:13.536 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:13.530+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:13.647 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:13.642+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:13.769 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:13.763+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:14.173 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:14.168+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:14.173 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:14.168+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:14.401 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:14.395+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:14.469 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:14.463+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:14.551 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:14.546+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:14.694 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:14.689+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:14.746 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:14.741+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:15.176 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:15.164+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:15.188 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:15.183+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:15.444 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:15.434+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:15.449 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:15.443+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:15.524 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:15.518+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:15.708 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:15.703+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:15.766 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:15.760+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:16.186 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:16.181+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:16.221 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:16.215+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:16.429 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:16.424+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:16.460 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:16.455+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:16.503 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:16.497+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:16.738 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:16.732+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:16.797 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:16.792+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:17.212 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:17.207+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:17.254 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:17.249+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:17.384 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:17.378+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:17.475 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:17.469+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:17.553 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:17.548+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:17.747 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:17.739+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:17.803 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:17.798+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:18.218 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:18.213+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:18.271 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:18.265+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:18.364 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:18.357+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:18.463 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:18.457+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:18.526 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:18.521+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:18.779 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:18.774+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:18.790 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:18.784+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:19.178 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:19.172+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:19.283 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:19.278+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:19.338 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:19.332+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:19.444 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:19.438+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:19.509 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:19.504+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:19.767 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:19.759+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:19.770 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:19.765+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:20.221 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:20.215+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:20.297 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:20.291+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:20.303 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:20.297+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:20.420 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:20.415+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:20.521 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:20.515+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:20.739 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:20.730+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:20.807 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:20.802+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:21.275 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:21.267+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:21.289 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:21.284+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:21.337 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:21.331+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:21.457 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:21.452+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:21.512 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:21.507+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:21.745 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:21.740+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:21.811 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:21.806+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:22.245 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:22.240+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:22.321 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:22.316+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:22.364 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:22.359+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:22.456 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:22.451+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:22.506 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:22.500+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:22.734 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:22.729+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:22.850 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:22.844+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:23.279 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:23.274+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:23.365 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:23.359+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:23.405 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:23.399+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:23.441 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:23.435+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:23.530 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:23.525+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:23.737 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:23.731+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:23.879 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:23.874+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:24.242 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:24.236+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:24.356 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:24.351+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:24.408 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:24.402+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:24.452 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:24.445+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:24.541 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:24.536+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:24.742 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:58:24.736+0000 1694f700 -1 osd.4 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:13.318553+0000 front 2019-12-09T17:57:13.298628+0000 (oldest deadline 2019-12-09T17:57:58.396172+0000) 2019-12-09T17:58:24.903 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:58:24.898+0000 1694f700 -1 osd.2 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.289263+0000 front 2019-12-09T17:57:17.259232+0000 (oldest deadline 2019-12-09T17:57:57.180063+0000) 2019-12-09T17:58:25.239 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:58:25.234+0000 1694f700 -1 osd.1 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.636882+0000 front 2019-12-09T17:57:16.309136+0000 (oldest deadline 2019-12-09T17:57:59.304930+0000) 2019-12-09T17:58:25.348 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:58:25.343+0000 1694f700 -1 osd.5 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:14.552969+0000 front 2019-12-09T17:57:15.484900+0000 (oldest deadline 2019-12-09T17:57:59.152805+0000) 2019-12-09T17:58:25.381 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:58:25.376+0000 1694f700 -1 osd.7 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:17.199255+0000 front 2019-12-09T17:57:17.207265+0000 (oldest deadline 2019-12-09T17:58:03.039911+0000) 2019-12-09T17:58:25.465 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:58:25.460+0000 1694f700 -1 osd.0 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:15.632586+0000 front 2019-12-09T17:57:15.444927+0000 (oldest deadline 2019-12-09T17:57:59.545592+0000) 2019-12-09T17:58:25.541 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T17:58:25.535+0000 1694f700 -1 osd.6 175 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T17:57:16.789898+0000 front 2019-12-09T17:57:17.103971+0000 (oldest deadline 2019-12-09T17:58:00.233556+0000) 2019-12-09T17:58:55.050 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:15:51.670 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.060 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:15:51.679 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.067 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:15:51.666 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.079 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:15:51.699 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.080 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:15:51.679 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.084 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:15:51.701 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.156 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:15:51.776 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.171 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:15:51.771 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T17:58:55.201 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:15:51.815 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:24.078 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:24.074+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:24.125 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:24.119+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:24.145 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:24.141+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:24.777 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:24.772+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:25.110 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:25.106+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:25.115 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:25.111+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:25.125 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:25.120+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:25.182 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:25.173+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:25.382 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:25.378+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:25.755 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:25.750+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:26.118 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:26.112+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:26.162 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:26.155+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:26.162 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:26.155+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:26.196 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:26.191+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:26.421 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:26.415+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:26.782 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:26.776+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:27.124 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:27.119+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:27.141 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:27.136+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:27.177 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:27.172+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:27.196 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:27.191+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:27.421 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:27.418+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:27.787 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:27.782+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:28.092 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:28.087+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:28.130 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:28.120+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:28.199 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:28.193+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:28.225 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:28.220+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:28.469 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:28.464+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:28.759 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:28.753+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:29.113 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:29.107+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:29.126 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:29.122+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:29.158 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:29.153+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:29.263 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:29.257+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:29.480 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:29.475+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:29.755 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:29.750+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:30.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:30.095+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:30.102 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:30.098+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:30.204 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:30.199+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:30.236 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:30.230+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:30.490 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:30.484+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:30.765 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:30.761+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:31.077 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:31.072+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:31.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:31.096+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:31.216 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:31.211+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:31.227 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:31.222+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:31.459 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:31.454+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:31.746 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:31.742+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:32.034 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:32.029+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:32.136 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:32.131+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:32.247 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:32.238+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:32.257 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:32.252+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:32.461 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:32.457+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:32.784 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:32.780+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:32.990 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:32.985+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:33.156 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:33.150+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:33.241 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:33.230+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:33.283 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:33.278+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:33.465 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:33.461+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:33.833 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:33.829+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:33.995 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:33.990+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:34.150 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:34.145+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:34.268 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:34.262+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:34.336 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:34.333+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:34.490 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:34.486+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:34.826 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:34.822+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:34.985 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:34.977+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:35.137 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:35.132+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:35.231 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:35.223+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:35.385 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:35.381+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:35.477 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:35.474+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:35.874 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:35.870+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:35.973 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:35.967+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:36.115 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:36.111+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:36.186 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:36.181+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:36.353 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:36.349+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:36.437 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:36.433+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:36.836 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:36.832+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:36.960 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:36.955+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:37.141 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:37.136+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:37.173 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:37.167+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:37.353 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:37.350+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:37.467 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:37.463+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:37.663 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:37.653+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:37.859 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:37.855+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:37.931 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:37.927+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:38.130 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:38.126+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:38.168 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:38.164+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:38.347 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:38.341+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:38.466 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:38.463+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:38.660 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:38.651+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:38.887 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:38.883+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:38.926 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:38.920+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:39.110 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:39.105+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:39.212 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:39.206+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:39.371 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:39.367+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:39.443 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:39.438+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:39.673 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:39.668+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:39.857 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:39.854+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:39.949 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:39.944+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:40.149 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:40.143+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:40.203 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:40.198+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:40.399 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:40.396+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:40.414 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:40.407+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:40.697 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:40.691+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:40.863 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:40.859+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:40.938 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:40.933+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:41.155 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:41.149+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:41.202 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:41.196+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:41.421 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:41.417+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:41.447 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:41.443+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:41.664 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:41.658+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:41.899 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:41.895+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:41.940 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:41.934+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:42.168 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:42.163+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:42.189 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:42.184+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:42.441 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:42.435+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:42.470 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:42.465+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:42.666 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:42.662+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:42.872 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:42.866+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:42.909 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:42.902+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:43.126 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:43.122+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:43.198 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:43.195+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:43.445 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:43.440+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:43.477 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:43.472+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:43.693 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:43.689+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:43.864 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:43.860+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:43.879 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:43.874+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:44.128 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:44.124+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:44.255 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:44.251+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:44.472 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:44.467+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:44.472 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:44.467+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:44.740 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:44.736+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:44.849 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:44.842+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:44.883 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:44.878+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:45.110 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:45.105+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:45.254 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:45.250+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:45.440 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:45.435+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:45.495 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:45.489+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:45.723 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:45.718+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:45.825 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:45.820+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:45.939 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:45.934+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:46.142 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:46.138+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:46.298 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:46.294+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:46.471 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:46.466+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:46.512 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:46.508+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:46.733 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:46.729+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:46.872 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:46.868+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:46.959 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:46.954+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:47.102 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:47.098+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:47.274 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:47.270+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:47.494 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:47.488+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:47.508 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:47.503+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:47.783 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:47.780+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:47.858 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:47.855+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:47.945 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:47.941+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:48.072 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:48.067+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:48.234 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:48.230+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:48.508 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:48.503+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:48.537 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:48.531+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:48.760 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:48.755+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:48.765 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:48.761+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:48.848 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:48.845+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:49.000 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:48.996+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:49.100 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:49.091+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:49.259 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:49.251+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:49.557 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:49.548+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:49.578 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:49.574+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:49.773 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:49.762+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:49.773 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:49.763+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:49.807 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:49.804+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:49.974 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:49.969+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:50.144 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:50.140+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:50.222 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:50.218+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:50.510 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:50.504+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:50.570 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:50.565+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:50.753 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:50.743+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:50.753 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:50.745+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:50.809 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:50.805+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:51.007 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:51.003+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:51.106 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:51.103+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:51.192 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:51.188+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:51.519 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:51.511+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:51.592 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:51.578+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:51.740 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:51.734+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:51.740 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:51.736+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:51.831 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:51.827+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:52.034 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:52.028+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:52.089 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T17:59:51.972+0000 20e25700 -1 RGWWatcher::handle_error cookie 821548144 err (107) Transport endpoint is not connected 2019-12-09T17:59:52.151 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:52.147+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:52.182 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:52.179+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:52.514 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:52.509+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:52.537 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:52.533+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:52.713 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:52.708+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:52.716 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:52.713+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:52.856 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:52.851+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:53.029 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:53.024+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:53.188 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:53.184+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:53.200 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:53.195+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:53.472 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:53.467+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:53.582 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:53.576+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:53.738 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:53.732+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:53.739 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:53.734+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:53.901 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:53.897+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:54.083 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:54.078+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:54.176 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:54.171+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:54.237 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:54.230+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:54.480 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:54.476+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:54.563 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:54.558+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:54.718 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:54.713+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:54.719 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:54.715+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:54.944 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:54.941+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:55.102 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:55.097+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:55.217 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:55.212+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:55.272 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:55.263+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:55.481 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:55.476+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:55.565 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:55.558+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:55.745 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:55.741+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:55.749 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:55.745+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:55.929 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:55.926+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:56.027 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T17:59:56.006+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e181 v181) 2019-12-09T17:59:56.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:56.056+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:56.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:56.059+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T17:59:56.228 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:56.223+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:56.306 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:56.302+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:56.534 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:56.529+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:56.588 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:56.583+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:56.760 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:56.756+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:56.763 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:56.760+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:56.903 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:56.899+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:57.054 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:57.048+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:57.055 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:57.051+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T17:59:57.213 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:57.208+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:57.357 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:57.352+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:57.565 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:57.560+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:57.592 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:57.588+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:57.815 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:57.811+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:57.818 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:57.814+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:57.891 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:57.886+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:58.107 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:58.102+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:58.109 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:58.105+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T17:59:58.258 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:58.254+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:58.351 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:58.347+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:58.614 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:58.609+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:58.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:58.619+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:58.834 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:58.820+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:58.835 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:58.822+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:58.891 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:58.887+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T17:59:59.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:59.157+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T17:59:59.168 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T17:59:59.158+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T17:59:59.230 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T17:59:59.226+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T17:59:59.349 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T17:59:59.334+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T17:59:59.586 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T17:59:59.580+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T17:59:59.625 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T17:59:59.620+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T17:59:59.796 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:59.791+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T17:59:59.796 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T17:59:59.793+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T17:59:59.849 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T17:59:59.845+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:00.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:00.162+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:00.170 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:00.166+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:00.235 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:00:00.231+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T18:00:00.324 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:00:00.320+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T18:00:00.631 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:00:00.626+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T18:00:00.662 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:00:00.657+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T18:00:00.806 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:00.801+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T18:00:00.807 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:00.803+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:00.890 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:00:00.886+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:01.027 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:00:01.022+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e181 v181) 2019-12-09T18:00:01.165 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:01.161+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:01.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:01.164+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:01.206 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:00:01.203+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T18:00:01.282 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:00:01.277+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T18:00:01.673 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:00:01.668+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T18:00:01.704 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:00:01.698+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T18:00:01.773 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:01.767+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T18:00:01.774 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:01.769+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:01.859 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:00:01.854+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:02.166 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:02.161+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:02.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:02.163+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:02.226 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:00:02.223+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T18:00:02.335 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:00:02.330+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T18:00:02.688 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:00:02.683+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T18:00:02.740 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:00:02.736+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T18:00:02.744 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:02.739+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T18:00:02.745 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:02.741+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:02.923 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:00:02.913+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:03.177 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:03.171+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:03.177 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:03.173+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:03.233 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:00:03.228+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T18:00:03.341 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:00:03.336+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T18:00:03.722 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:03.718+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T18:00:03.723 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:03.719+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:03.740 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:00:03.734+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T18:00:03.747 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:00:03.742+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T18:00:03.932 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:00:03.928+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:04.163 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:04.158+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:04.164 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:04.159+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:04.224 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:00:04.220+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T18:00:04.388 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:00:04.384+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T18:00:04.722 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:04.719+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T18:00:04.725 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:04.722+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:04.764 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:00:04.748+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T18:00:04.773 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:00:04.768+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T18:00:04.962 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:00:04.957+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:05.184 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:05.179+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:05.184 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:05.180+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:05.265 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:00:05.261+0000 1694f700 -1 osd.2 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:39.623325+0000 front 2019-12-09T17:58:39.624398+0000 (oldest deadline 2019-12-09T17:59:23.588605+0000) 2019-12-09T18:00:05.373 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:00:05.369+0000 1694f700 -1 osd.1 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.111780+0000 front 2019-12-09T17:58:43.613115+0000 (oldest deadline 2019-12-09T17:59:24.608482+0000) 2019-12-09T18:00:05.685 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:05.681+0000 1694f700 -1 osd.3 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T17:58:57.333893+0000 (oldest deadline 2019-12-09T17:59:37.333893+0000) 2019-12-09T18:00:05.689 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:05.685+0000 1694f700 -1 osd.3 181 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:05.769 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:00:05.764+0000 1694f700 -1 osd.5 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:42.648167+0000 front 2019-12-09T17:58:43.627818+0000 (oldest deadline 2019-12-09T17:59:23.488130+0000) 2019-12-09T18:00:05.793 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:00:05.789+0000 1694f700 -1 osd.7 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:40.749267+0000 front 2019-12-09T17:58:40.753978+0000 (oldest deadline 2019-12-09T17:59:24.874256+0000) 2019-12-09T18:00:05.942 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:00:05.938+0000 1694f700 -1 osd.0 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:43.969913+0000 front 2019-12-09T17:58:43.629388+0000 (oldest deadline 2019-12-09T17:59:23.444149+0000) 2019-12-09T18:00:06.071 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:00:06.068+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e181 v181) 2019-12-09T18:00:06.236 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:06.232+0000 1694f700 -1 osd.4 181 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T17:58:38.561817+0000 front 2019-12-09T17:58:38.559525+0000 (oldest deadline 2019-12-09T17:59:24.387647+0000) 2019-12-09T18:00:06.237 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:06.233+0000 1694f700 -1 osd.4 181 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:00:07.010 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:00:06.878+0000 1694f700 -1 osd.3 182 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:118412 2.2fs0 2.e6b4096f (undecoded) ondisk+retry+write+known_if_redirected e179) 2019-12-09T18:00:07.296 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:00:07.288+0000 1694f700 -1 osd.4 182 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:118826 5.3 5:c609908c:::notify.5:head [watch ping cookie 821548144 gen 66] snapc 0=[] ondisk+write+known_if_redirected e181) 2019-12-09T18:02:10.108 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:10.099+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:10.319 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:10.312+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:11.068 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:11.064+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:11.355 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:11.349+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:11.540 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:11.536+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:12.102 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:12.098+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:12.326 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:12.321+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:12.540 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:12.534+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:12.930 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:12.925+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:13.096 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:13.089+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:13.311 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:13.305+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:13.331 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:13.326+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:13.517 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:13.511+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:13.924 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:13.919+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:14.076 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:14.070+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:14.102 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:14.096+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:14.290 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:14.285+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:14.330 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:14.325+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:14.497 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:14.491+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:14.899 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:14.892+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:15.071 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:15.066+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:15.101 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:15.095+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:15.105 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:15.099+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:15.301 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:15.295+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:15.376 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:15.369+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:15.466 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:15.460+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:15.861 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:15.856+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:16.050 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:16.045+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:16.072 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:16.066+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:16.079 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:16.073+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:16.354 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:16.347+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:16.368 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:16.362+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:16.438 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:16.429+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:16.823 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:16.818+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:17.033 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:17.026+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:17.041 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:17.036+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:17.085 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:17.080+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:17.368 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:17.363+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:17.416 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:17.410+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:17.461 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:17.455+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:17.843 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:17.839+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:18.039 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:18.033+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:18.040 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:18.034+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:18.096 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:18.092+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:18.398 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:18.394+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:18.455 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:18.449+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:18.516 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:18.510+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:18.874 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:18.869+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:19.076 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:19.070+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:19.101 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:19.092+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:19.107 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:19.103+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:19.366 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:19.352+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:19.438 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:19.433+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:19.543 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:19.538+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:19.889 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:19.879+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:20.081 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:20.072+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:20.087 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:20.081+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:20.116 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:20.111+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:20.384 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:20.379+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:20.398 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:20.393+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:20.530 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:20.525+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:20.848 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:20.843+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:21.058 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:21.053+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:21.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:21.128+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:21.137 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:21.131+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:21.391 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:21.386+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:21.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:21.395+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:21.545 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:21.540+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:21.846 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:21.842+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:22.103 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:22.099+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:22.147 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:22.141+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:22.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:22.161+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:22.357 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:22.345+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:22.365 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:22.360+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:22.535 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:22.530+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:22.868 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:22.863+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:23.126 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:23.120+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:23.137 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:23.131+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:23.169 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:23.159+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:23.373 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:23.359+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:23.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:23.362+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:23.523 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:23.517+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:23.848 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:23.844+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:24.083 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:24.077+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:24.110 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:24.106+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:24.128 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:24.122+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:24.338 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:24.333+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:24.343 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:24.338+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:24.546 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:24.540+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:24.902 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:24.896+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:25.087 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:25.082+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:25.088 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:25.083+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:25.134 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:25.128+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:25.325 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:25.319+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:25.366 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:25.361+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:25.601 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:25.596+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:25.876 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:25.870+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:26.074 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:26.067+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:26.138 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:26.134+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:26.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:26.151+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:26.328 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:26.322+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:26.363 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:26.359+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:26.628 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:26.622+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:26.833 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:26.829+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:27.111 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:27.105+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:27.153 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:27.147+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:27.156 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:27.151+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:27.306 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:27.301+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:27.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:27.399+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:27.666 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:27.662+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:27.883 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:27.879+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:28.110 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:28.106+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:28.134 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:28.127+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:28.202 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:28.197+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:28.349 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:28.344+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:28.438 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:28.433+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:28.689 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:28.682+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:28.878 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:28.875+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:29.089 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:29.086+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:29.160 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:29.155+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:29.243 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:29.237+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:29.345 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:29.339+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:29.427 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:29.422+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:29.701 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:29.696+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:29.889 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:29.883+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:30.123 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:30.117+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:30.127 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:30.122+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:30.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:30.267+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:30.325 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:30.319+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:30.446 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:30.441+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:30.743 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:30.737+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:30.844 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:30.839+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:31.133 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:31.128+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:31.136 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:31.130+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:31.309 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:31.298+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:31.319 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:31.314+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:31.436 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:31.432+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:31.731 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:31.726+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:31.797 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:31.793+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:32.124 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:32.119+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:32.182 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:32.178+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:32.321 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:32.315+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:32.322 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:32.317+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:32.467 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:32.462+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:32.756 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:32.751+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:32.769 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:32.765+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:33.130 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:33.125+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:33.183 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:33.178+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:33.360 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:33.351+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:33.368 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:33.363+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:33.466 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:33.462+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:33.785 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:33.781+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:33.806 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:33.802+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:34.118 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:34.114+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:34.205 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:34.200+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:34.362 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:34.356+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:34.411 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:34.406+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:34.480 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:34.475+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:34.747 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:34.742+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:34.846 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:34.842+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:35.085 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:35.080+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:35.200 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:35.195+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:35.355 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:35.351+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:35.436 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:35.431+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:35.525 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:35.521+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:35.752 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:35.747+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:35.866 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:35.861+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:36.045 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:36.038+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:36.243 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:36.239+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:36.398 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:36.393+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:36.464 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:36.458+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:36.485 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:36.481+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:36.743 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:36.739+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:36.838 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:36.832+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:37.045 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:37.039+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:37.232 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:37.227+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:37.367 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:37.363+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:37.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:37.464+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:37.520 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:37.513+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:37.734 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:37.729+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:37.856 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:37.852+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:38.045 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:38.041+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:38.217 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:38.213+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:38.418 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:38.413+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:38.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:38.477+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:38.514 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:38.509+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:38.761 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:38.757+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:38.835 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:38.831+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:39.094 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:39.090+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:39.200 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:39.196+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:39.415 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:39.410+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:39.467 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:39.462+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:39.559 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:39.553+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:39.784 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:39.780+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:39.870 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:39.866+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:40.076 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:40.070+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:40.205 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:40.201+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:40.440 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:40.435+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:40.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:40.474+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:40.577 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:40.572+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:40.821 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:40.816+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:40.925 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:40.920+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:41.056 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:41.052+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:41.204 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:41.198+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:41.465 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:41.460+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:41.492 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:41.486+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:41.538 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:02:41.534+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e187 v187) 2019-12-09T18:02:41.553 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:41.548+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:41.857 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:41.853+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:41.941 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:41.935+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:42.037 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:42.027+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:42.235 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:42.229+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:42.450 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:42.445+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:42.502 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:42.497+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:42.549 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:42.544+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:42.851 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:42.846+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:42.934 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:42.929+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:43.065 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:43.061+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:43.209 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:43.204+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:43.454 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:43.448+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:43.522 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:43.516+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:43.527 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:43.523+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:43.892 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:43.889+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:43.930 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:43.924+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:44.113 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:44.108+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:44.201 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:44.190+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:44.454 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:44.448+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:44.540 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:44.535+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:44.566 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:44.560+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:44.863 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:44.857+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:44.933 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:44.927+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:45.128 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:45.123+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:45.165 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:45.159+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:45.437 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:45.431+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:45.563 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:45.557+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:45.571 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:45.565+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:45.821 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:02:45.815+0000 1694f700 -1 osd.1 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.045910+0000 front 2019-12-09T18:01:28.940800+0000 (oldest deadline 2019-12-09T18:02:12.387823+0000) 2019-12-09T18:02:45.957 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:45.952+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:46.096 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:02:46.091+0000 1694f700 -1 osd.2 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:24.004107+0000 front 2019-12-09T18:01:24.014332+0000 (oldest deadline 2019-12-09T18:02:09.862307+0000) 2019-12-09T18:02:46.132 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:02:46.127+0000 1694f700 -1 osd.5 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.526529+0000 front 2019-12-09T18:01:29.532082+0000 (oldest deadline 2019-12-09T18:02:14.796087+0000) 2019-12-09T18:02:46.410 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:02:46.404+0000 1694f700 -1 osd.6 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.937272+0000 front 2019-12-09T18:01:27.410080+0000 (oldest deadline 2019-12-09T18:02:09.679297+0000) 2019-12-09T18:02:46.593 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:02:46.586+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e187 v187) 2019-12-09T18:02:46.603 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:02:46.598+0000 1694f700 -1 osd.0 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:27.936751+0000 front 2019-12-09T18:01:27.834064+0000 (oldest deadline 2019-12-09T18:02:12.822569+0000) 2019-12-09T18:02:46.614 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:02:46.608+0000 1694f700 -1 osd.4 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:29.618802+0000 front 2019-12-09T18:01:29.212803+0000 (oldest deadline 2019-12-09T18:02:13.238733+0000) 2019-12-09T18:02:46.939 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:02:46.933+0000 1694f700 -1 osd.7 187 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:01:28.167482+0000 front 2019-12-09T18:01:27.924691+0000 (oldest deadline 2019-12-09T18:02:10.774443+0000) 2019-12-09T18:02:56.162 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:19:52.777 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.166 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:19:52.783 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.170 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:19:52.789 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.173 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:19:52.793 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.185 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:19:52.784 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.193 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:19:52.792 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.418 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:19:53.038 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.429 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:19:53.049 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:56.477 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:19:53.071 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:30.132 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:30.128+0000 1694f700 -1 osd.3 189 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4367.0:130746 7.7 7:e31d3d5f:lc::lc.28:head [call lock.lock in=50b] snapc 0=[] RETRY=14 ondisk+retry+write+known_if_redirected e184) 2019-12-09T18:03:32.403 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.385+0000 34274700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:32.527 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.451+0000 34274700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:32.619 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.493+0000 34a75700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:32.671 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.562+0000 34274700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:32.771 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.603+0000 3727a700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:32.799 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.653+0000 37a7b700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:32.833 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:32.725+0000 114ae700 -1 osd.4 191 build_incremental_map_msg missing incremental map 191 2019-12-09T18:03:34.524 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:34.514+0000 36a79700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:34.954 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:34.857+0000 3827c700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:35.104 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:34.913+0000 3827c700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:35.189 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:34.931+0000 36a79700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:35.221 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:35.052+0000 34274700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:35.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:35.062+0000 35a77700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:35.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:35.079+0000 3aa81700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:35.335 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:35.175+0000 3aa81700 -1 osd.3 192 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:36.739 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:36.727+0000 34a75700 -1 osd.3 193 build_incremental_map_msg missing incremental map 193 2019-12-09T18:03:38.567 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:38.562+0000 3aa81700 -1 osd.3 193 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:38.571 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:38.563+0000 3aa81700 -1 osd.3 193 build_incremental_map_msg missing incremental map 193 2019-12-09T18:03:39.279 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:39.262+0000 3b282700 -1 osd.3 193 build_incremental_map_msg missing incremental map 192 2019-12-09T18:03:39.286 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:03:39.263+0000 3b282700 -1 osd.3 193 build_incremental_map_msg missing incremental map 193 2019-12-09T18:03:42.809 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:42.804+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:43.852 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:43.847+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:43.922 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:43.917+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:44.838 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:44.833+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:44.915 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:44.911+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:44.983 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:44.976+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:45.821 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:45.816+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:45.963 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:45.959+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:46.034 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:46.024+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:46.602 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:46.598+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:46.832 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:46.828+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:46.930 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:46.926+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:47.050 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:47.045+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:47.063 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:47.059+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:47.623 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:47.616+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:47.810 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:47.806+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:47.937 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:47.931+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:48.087 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:48.083+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:48.088 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:48.084+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:48.590 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:48.585+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:48.607 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:48.602+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:48.816 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:48.812+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:48.895 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:48.890+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:49.068 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:49.063+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:49.072 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:49.066+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:49.569 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:49.565+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:49.607 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:49.603+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:49.868 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:49.861+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:49.910 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:49.906+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:50.085 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:50.078+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:50.118 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:50.114+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:50.586 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:50.580+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:50.635 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:50.630+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:50.876 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:50.871+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:50.880 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:50.875+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:51.059 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:51.054+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:51.122 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:51.116+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:51.568 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:51.564+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:51.670 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:51.664+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:51.859 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:51.854+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:51.869 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:51.857+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:52.092 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:52.087+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:52.155 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:52.150+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:52.602 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:52.597+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:52.673 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:52.669+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:52.831 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:52.826+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:52.913 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:52.909+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:53.144 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:53.139+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:53.147 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:53.142+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:53.609 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:53.604+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:53.664 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:53.659+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:53.855 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:53.851+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:53.910 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:53.906+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:54.103 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:54.097+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:54.104 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:54.099+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:54.621 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:54.614+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:54.675 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:54.669+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:54.838 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:54.834+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:54.923 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:54.919+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:55.069 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:55.064+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:55.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:55.152+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:55.610 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:55.606+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:55.651 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:55.646+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:55.846 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:55.841+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:55.885 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:55.881+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:56.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:56.089+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:56.208 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:56.203+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:56.623 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:56.618+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:56.685 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:56.680+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:56.876 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:56.871+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:56.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:56.873+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:57.096 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:57.091+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:57.249 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:57.244+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:57.595 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:57.590+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:57.692 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:57.687+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:57.895 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:57.890+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:57.906 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:57.901+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:58.081 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:58.076+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:58.283 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:58.277+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:58.604 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:58.599+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:58.692 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:58.687+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:58.917 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:58.913+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:58.946 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:58.942+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:03:59.119 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:03:59.114+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:03:59.253 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:03:59.248+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:03:59.642 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:03:59.632+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:03:59.669 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:03:59.665+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:03:59.930 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:03:59.924+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:03:59.985 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:03:59.981+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:00.150 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:00.144+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:00.210 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:00.205+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:00.620 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:00.615+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:00.673 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:00.668+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:00.932 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:00.927+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:00.978 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:00.972+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:01.195 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:01.190+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:01.248 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:01.244+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:01.622 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:01.617+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:01.713 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:01.709+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:01.891 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:01.885+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:01.951 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:01.947+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:02.214 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:02.210+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:02.231 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:02.227+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:02.546 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:04:02.493+0000 20e25700 -1 RGWWatcher::handle_error cookie 5400794000 err (107) Transport endpoint is not connected 2019-12-09T18:04:02.581 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:02.578+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:02.719 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:02.714+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:02.869 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:02.864+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:02.987 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:02.981+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:03.204 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:03.199+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:03.238 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:03.234+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:03.541 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:03.538+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:03.726 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:03.719+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:03.828 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:03.823+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:04.022 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:04.017+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:04.038 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:04.034+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:04.208 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:04.204+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:04.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:04.269+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:04.274 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:04.270+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:04.536 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:04.532+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:04.722 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:04.716+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:04.859 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:04.853+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:05.032 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:05.026+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:05.064 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:05.058+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:05.209 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:05.205+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:05.230 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:05.223+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:05.230 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:05.225+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:05.496 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:05.492+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:05.712 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:05.707+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:05.834 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:05.830+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:06.060 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:06.056+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:06.077 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:06.073+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:06.243 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:06.240+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:06.246 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:06.242+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:06.246 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:06.242+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:06.481 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:06.478+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:06.723 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:06.717+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:06.879 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:06.873+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:07.058 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:07.053+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:07.072 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:07.065+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:07.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:07.214+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:07.222 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:07.219+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:07.225 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:07.223+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:07.471 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:07.466+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:07.756 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:07.750+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:07.929 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:07.922+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:08.036 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:08.030+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:08.101 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:08.095+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:08.257 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:08.253+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:08.275 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:08.271+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:08.276 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:08.272+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:08.515 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:08.510+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:08.795 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:08.789+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:08.909 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:08.903+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:09.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:09.037+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:09.128 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:09.122+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:09.269 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:09.264+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:09.269 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:09.266+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:09.291 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:09.286+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:09.491 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:09.487+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:09.831 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:09.826+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:09.949 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:09.943+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:10.043 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:10.038+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:10.119 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:10.113+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:10.267 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:10.261+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:10.267 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:10.262+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:10.288 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:10.283+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:10.501 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:10.496+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:10.818 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:10.813+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:10.906 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:10.901+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:11.079 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:11.070+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:11.079 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:11.072+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:11.124 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:11.119+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:11.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:11.254+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:11.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:11.255+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:11.268 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:11.263+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:11.500 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:11.496+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:11.856 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:11.851+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:11.918 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:11.913+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:12.053 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:12.047+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:12.053 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:12.049+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:12.158 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:12.153+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:12.261 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:12.251+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:12.275 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:12.270+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:12.275 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:12.272+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:12.461 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:12.456+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:12.832 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:12.826+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:12.948 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:12.942+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:13.060 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:13.053+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:13.061 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:13.055+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:13.203 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:13.198+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:13.288 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:13.283+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:13.288 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:13.284+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:13.305 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:13.300+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:13.457 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:13.452+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:13.863 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:13.857+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:13.968 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:13.962+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:14.037 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:14.031+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:14.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:14.035+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:14.206 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:14.199+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:14.301 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:14.297+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:14.305 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:14.301+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:14.361 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:14.356+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:14.491 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:14.487+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:14.849 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:14.843+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:14.940 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:14.935+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:15.003 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:14.997+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:15.004 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:14.998+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:15.183 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:15.178+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:15.326 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:15.322+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:15.330 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:15.326+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:15.342 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:15.336+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:15.491 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:15.486+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:15.857 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:15.851+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:15.923 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:15.917+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:16.016 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:16.011+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:16.019 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:16.014+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:16.210 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:16.204+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:16.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:16.343+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:16.356 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:16.348+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:16.363 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:16.357+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:16.518 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:16.513+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:16.841 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:16.836+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:16.884 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:16.879+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:17.006 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:16.997+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:17.025 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:17.015+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:17.025 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:17.017+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:17.197 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:17.192+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:17.375 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:17.370+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:17.379 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:17.374+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:17.395 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:17.390+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:17.489 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:17.484+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:17.812 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:17.807+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:17.907 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:17.902+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:17.977 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:17.971+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:17.978 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:17.973+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:18.157 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:18.151+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:18.351 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:18.345+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:18.393 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:18.389+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:18.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:18.391+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:18.447 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:18.442+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:18.856 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:18.851+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:18.949 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:18.944+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:18.993 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:18.988+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:18.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:18.992+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:19.180 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:19.174+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:19.320 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:19.314+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:19.396 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:19.388+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:19.397 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:19.392+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:19.450 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:19.445+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:19.870 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:19.865+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:19.907 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:19.902+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:20.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:20.033+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:20.042 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:20.037+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:20.190 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:20.184+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:20.342 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:20.334+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:20.377 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:20.366+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:20.377 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:20.367+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:20.488 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:20.484+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:20.856 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:20.850+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:20.893 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:20.886+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:21.088 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:21.083+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:21.089 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:21.084+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:21.208 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:21.202+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:21.352 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:21.347+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:21.355 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:21.350+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:21.356 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:21.352+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:21.485 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:21.481+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:21.822 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:21.818+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:21.901 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:21.895+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:22.023 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:22.017+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:22.137 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:22.131+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:22.138 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:22.132+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:22.165 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:22.160+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:22.393 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:22.388+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:22.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:22.390+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:22.396 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:22.391+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:22.487 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:04:22.483+0000 1694f700 -1 osd.7 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.882153+0000 front 2019-12-09T18:03:06.025375+0000 (oldest deadline 2019-12-09T18:03:48.190560+0000) 2019-12-09T18:04:22.803 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:22.797+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:22.868 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:22.863+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:23.124 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:23.118+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:23.124 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:23.119+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:23.184 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:23.178+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:23.397 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:23.392+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:23.399 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:23.395+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:23.424 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:23.419+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:23.806 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:23.801+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:23.893 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:23.887+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:24.163 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:24.156+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:24.163 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:24.158+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:24.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:24.193+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:24.403 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:24.398+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:24.419 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:24.414+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:24.422 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:24.418+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:24.827 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:24.822+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:24.936 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:24.931+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:25.119 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:25.112+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:25.120 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:25.114+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:25.256 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:25.249+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:25.409 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:25.405+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:25.459 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:25.455+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:25.460 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:25.456+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:25.793 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:25.787+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:25.909 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:25.904+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:26.139 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:26.132+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:26.139 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:26.134+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:26.277 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:26.271+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:26.411 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:26.407+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:26.423 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:26.419+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:26.437 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:26.422+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:26.839 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:26.827+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:26.895 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:26.889+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:27.043 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:27.037+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:27.162 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:27.157+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:27.162 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:27.158+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:27.323 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:27.318+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:27.369 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:27.364+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:27.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:27.377+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:27.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:27.379+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:27.826 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:27.820+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:27.886 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:27.881+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:28.123 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:28.117+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:28.123 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:28.118+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:28.350 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:28.345+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:28.369 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:28.363+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:28.376 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:28.371+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:28.379 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:28.375+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:28.787 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:28.781+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:28.878 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:28.873+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:29.172 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:29.165+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:29.173 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:29.167+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:29.328 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:29.322+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:29.400 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:29.395+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:29.405 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:29.399+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:29.405 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:29.401+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:29.790 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:29.784+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:29.878 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:29.873+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:30.136 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:30.131+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:30.139 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:30.134+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:30.314 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:30.307+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:30.407 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:30.399+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:30.410 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:30.404+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:30.449 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:30.444+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:30.812 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:30.807+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:30.877 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:30.871+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:31.122 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:31.117+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:31.123 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:31.118+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:31.309 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:31.304+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:31.409 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:31.400+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:31.463 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:31.457+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:31.464 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:31.459+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:31.798 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:31.793+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:31.883 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:31.877+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:32.062 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:32.052+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:32.141 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:32.135+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:32.142 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:32.137+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:32.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:32.301+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:32.424 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:32.420+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:32.459 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:32.454+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:32.461 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:32.457+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:32.828 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:32.822+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:32.877 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:32.871+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:33.172 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:33.166+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:33.173 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:33.168+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:33.313 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:33.307+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:33.470 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:33.465+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:33.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:33.474+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:33.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:33.477+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:33.867 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:33.862+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:33.883 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:33.873+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:34.164 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:34.159+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:34.166 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:34.161+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:34.307 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:34.302+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:34.480 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:34.474+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:34.495 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:34.491+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:34.500 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:34.496+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:34.834 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:34.829+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:34.917 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:34.912+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:35.211 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:35.206+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:35.214 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:35.210+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:35.325 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:35.320+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:35.508 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:35.503+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:35.536 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:35.532+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:35.538 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:35.534+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:35.812 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:35.806+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:35.950 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:35.946+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:36.208 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:36.200+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:36.216 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:36.202+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:36.289 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:36.283+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:36.531 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:36.527+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:36.540 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:36.534+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:36.541 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:36.536+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:36.863 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:36.858+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:36.986 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:36.978+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:37.079 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:37.073+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:37.227 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:37.221+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:37.227 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:37.223+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:37.333 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:37.327+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:37.507 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:37.500+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:37.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:37.542+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:37.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:37.544+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:37.856 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:37.851+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:37.962 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:37.957+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:38.260 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:38.255+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:38.261 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:38.256+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:38.291 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:38.285+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:38.536 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:38.531+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:38.538 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:38.534+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:38.540 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:38.536+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:38.858 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:38.852+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:39.007 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:39.002+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:39.271 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:39.266+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:39.279 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:39.273+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:39.280 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:39.275+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:39.541 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:39.534+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:39.542 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:39.537+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:39.567 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:39.562+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:39.901 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:39.896+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:39.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:39.970+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:40.237 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:40.230+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:40.238 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:40.232+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:40.284 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:40.278+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:40.508 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:40.501+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:40.510 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:40.504+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:40.524 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:40.519+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:40.936 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:40.931+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:40.983 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:40.978+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:41.252 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:41.246+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:41.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:41.248+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:41.324 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:41.319+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:41.465 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:41.459+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:41.466 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:41.461+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:41.564 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:41.559+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:41.900 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:41.895+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:41.969 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:41.964+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:42.098 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:42.092+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:42.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:42.244+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:42.250 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:42.246+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:42.343 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:42.337+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:42.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:42.473+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:42.485 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:42.480+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:42.542 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:42.537+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:42.922 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:42.916+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:42.984 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:42.978+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:43.269 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:43.263+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:43.269 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:43.265+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:43.369 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:43.363+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:43.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:43.473+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:43.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:43.475+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:43.555 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:43.544+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:43.959 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:43.953+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:44.027 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:44.021+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:44.259 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:44.250+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:44.259 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:44.253+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:44.340 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:44.329+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:44.501 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:44.495+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:44.502 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:44.497+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:44.604 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:44.597+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:44.935 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:44.927+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:45.078 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:45.073+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:45.227 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:45.222+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:45.230 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:45.225+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:45.357 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:45.351+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:45.532 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:45.526+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:45.533 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:45.529+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:45.604 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:45.600+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:45.901 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:45.896+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:46.113 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:46.107+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:46.196 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:46.191+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:46.200 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:46.195+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:46.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:04:46.398+0000 1694f700 -1 osd.0 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.615098+0000 front 2019-12-09T18:03:07.225762+0000 (oldest deadline 2019-12-09T18:03:43.142961+0000) 2019-12-09T18:04:46.547 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:46.542+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:46.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:46.544+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:46.598 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:46.593+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:46.858 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:04:46.852+0000 1694f700 -1 osd.2 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:03.879832+0000 front 2019-12-09T18:03:00.821185+0000 (oldest deadline 2019-12-09T18:03:42.113862+0000) 2019-12-09T18:04:47.174 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:04:47.170+0000 1694f700 -1 osd.1 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:06.019002+0000 front 2019-12-09T18:03:06.020241+0000 (oldest deadline 2019-12-09T18:03:46.060749+0000) 2019-12-09T18:04:47.189 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:04:47.183+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e193 v193) 2019-12-09T18:04:47.203 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:47.198+0000 1694f700 -1 osd.3 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:03:30.101875+0000 (oldest deadline 2019-12-09T18:04:10.101875+0000) 2019-12-09T18:04:47.205 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:04:47.201+0000 1694f700 -1 osd.3 193 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e191) 2019-12-09T18:04:47.529 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:47.525+0000 1694f700 -1 osd.4 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:04.354543+0000 front 2019-12-09T18:03:03.859046+0000 (oldest deadline 2019-12-09T18:03:46.952331+0000) 2019-12-09T18:04:47.530 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:47.526+0000 1694f700 -1 osd.4 193 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:04:47.554 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:04:47.551+0000 1694f700 -1 osd.5 193 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:03:02.437568+0000 front 2019-12-09T18:03:02.440537+0000 (oldest deadline 2019-12-09T18:03:44.284564+0000) 2019-12-09T18:04:48.566 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:04:48.562+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119290 5.3 5:c609908c:::notify.5:head [watch ping cookie 5400794000 gen 2] snapc 0=[] ondisk+write+known_if_redirected e191) 2019-12-09T18:05:13.902 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:13.897+0000 36a79700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.009 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.001+0000 3aa81700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.065 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.034+0000 3927e700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.100 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.039+0000 36278700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.142 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.070+0000 3927e700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.156 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.122+0000 33a73700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.180 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.153+0000 3b282700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.208 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.204+0000 3b282700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.224 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.210+0000 35276700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:05:14.282 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:05:14.210+0000 36a79700 -1 osd.6 199 build_incremental_map_msg missing incremental map 199 2019-12-09T18:06:12.588 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:23:09.207 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.629 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:23:09.249 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.687 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:23:09.307 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.691 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:23:09.310 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.720 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:23:09.318 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.725 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:23:09.323 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.847 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:23:09.467 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.923 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:23:09.543 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:12.937 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:23:09.536 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:06:30.105 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:30.100+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:30.627 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:30.623+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:30.883 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:30.877+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:31.152 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:31.148+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:31.360 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:31.354+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:31.578 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:31.573+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:31.655 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:31.650+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:31.871 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:31.865+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:32.130 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:32.125+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:32.312 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:32.306+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:32.339 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:32.333+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:32.629 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:32.625+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:32.651 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:32.646+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:32.902 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:32.895+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:33.101 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:33.095+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:33.331 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:33.326+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:33.361 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:33.355+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:33.661 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:33.657+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:33.666 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:33.662+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:33.890 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:33.883+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:34.089 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:34.084+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:34.106 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:34.101+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:34.346 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:34.341+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:34.362 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:34.354+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:34.638 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:34.633+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:34.697 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:34.693+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:34.933 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:34.925+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:35.056 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:35.051+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:35.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:35.083+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:35.329 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:35.322+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:35.380 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:35.375+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:35.620 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:35.615+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:35.667 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:35.662+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:35.981 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:35.975+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:36.075 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:36.070+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:36.076 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:36.072+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:36.370 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:36.365+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:36.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:36.367+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:36.631 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:36.625+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:36.711 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:36.701+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:36.957 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:36.952+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:37.032 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:37.027+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:37.076 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:37.071+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:37.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:37.328+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:37.422 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:37.415+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:37.616 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:37.610+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:37.719 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:37.715+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:37.981 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:37.975+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:38.030 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:38.023+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:38.043 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:38.037+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:38.309 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:38.303+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:38.432 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:38.426+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:38.616 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:38.611+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:38.717 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:38.709+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:38.983 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:38.977+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:39.026 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:39.021+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:39.067 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:39.060+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:39.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:39.275+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:39.471 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:39.465+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:39.588 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:39.582+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:39.769 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:39.763+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:40.026 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:40.020+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:40.027 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:40.021+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:40.050 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:40.045+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:40.284 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:40.279+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:40.506 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:40.500+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:40.561 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:40.553+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:40.798 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:40.792+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:41.004 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:40.998+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:41.032 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:41.025+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:41.091 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:41.084+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:41.330 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:41.324+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:41.497 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:41.491+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:41.591 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:41.585+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:41.795 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:41.789+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:42.029 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:42.023+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:42.064 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:42.059+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:42.131 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:42.126+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:42.306 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:42.298+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:42.483 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:42.477+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:42.564 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:42.558+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:42.775 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:42.770+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:43.065 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:43.059+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:43.078 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:43.067+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:43.165 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:43.160+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:43.263 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:43.258+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:43.512 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:43.506+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:43.595 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:43.589+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:43.797 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:43.792+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:44.036 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:44.030+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:44.101 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:44.095+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:44.220 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:44.214+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:44.251 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:44.245+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:44.559 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:44.552+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:44.626 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:44.620+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:44.783 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:44.777+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:45.023 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:45.017+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:45.105 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:45.100+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:45.229 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:45.223+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:45.231 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:45.224+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:45.565 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:45.559+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:45.682 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:45.675+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:45.842 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:45.837+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:46.045 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:46.038+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:46.130 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:46.125+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:46.188 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:46.182+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:46.192 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:46.187+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:46.600 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:46.593+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:46.733 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:46.727+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:46.861 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:46.855+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:47.088 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:47.082+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:47.174 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:47.168+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:47.204 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:47.197+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:47.239 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:47.233+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:47.582 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:47.576+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:47.753 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:47.746+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:47.882 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:47.875+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:48.100 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:48.094+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:48.178 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:48.172+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:48.220 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:48.214+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:48.228 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:48.222+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:48.540 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:48.534+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:48.789 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:48.784+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:48.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:48.846+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:49.103 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:49.096+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:49.213 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:49.207+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:49.228 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:49.222+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:49.255 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:49.249+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:49.511 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:49.506+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:49.770 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:49.764+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:49.853 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:49.843+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:50.131 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:50.125+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:50.244 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:50.237+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:50.256 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:50.250+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:50.266 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:50.260+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:50.553 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:50.547+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:50.812 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:50.805+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:50.816 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:50.811+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:51.097 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:51.091+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:51.263 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:51.257+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:51.294 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:51.288+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:51.313 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:51.307+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:51.545 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:51.540+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:51.811 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:51.806+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:51.823 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:51.817+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:52.073 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:52.066+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:52.249 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:52.244+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:52.250 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:52.245+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:52.302 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:52.296+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:52.588 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:52.578+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:52.828 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:52.822+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:52.844 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:52.837+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:53.088 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:53.082+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:53.261 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:53.252+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:53.290 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:53.285+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:53.300 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:53.294+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:53.581 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:53.576+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:53.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:53.846+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:53.855 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:53.848+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:54.118 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:54.111+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:54.258 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:54.252+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:54.308 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:54.302+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:54.313 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:54.307+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:54.579 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:54.573+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:54.828 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:54.822+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:54.881 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:54.875+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:55.082 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:55.076+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:55.274 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:55.267+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:55.277 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:55.272+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:55.306 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:55.299+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:55.597 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:55.591+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:55.817 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:55.810+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:55.925 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:55.919+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:56.061 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:56.055+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:56.281 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:56.271+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:56.320 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:56.311+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:56.349 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:56.343+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:56.618 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:56.612+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:56.837 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:56.830+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:56.920 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:56.913+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:57.073 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:57.067+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:57.326 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:57.320+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:57.368 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:57.355+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:57.399 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:57.392+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:57.585 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:57.579+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:57.839 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:57.833+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:57.953 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:57.948+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:58.100 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:58.094+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:58.297 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:58.290+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:58.361 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:58.355+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:58.455 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:58.448+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:58.568 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:58.563+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:58.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:58.884+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:59.006 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:58.999+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:06:59.127 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:06:59.123+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:06:59.291 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:06:59.286+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:06:59.379 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:06:59.374+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:06:59.429 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:06:59.422+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:06:59.611 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:06:59.607+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:06:59.914 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:06:59.908+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:06:59.969 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:06:59.962+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:00.102 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:00.097+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:00.264 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:00.259+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:00.424 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:00.416+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:00.431 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:00.427+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:00.664 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:00.659+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:00.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:00.882+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:00.966 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:00.959+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:01.152 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:01.148+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:01.224 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:01.220+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:01.410 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:01.403+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:01.428 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:01.423+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:01.706 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:01.701+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:01.928 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:01.921+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:02.006 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:02.001+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:02.148 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:02.143+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:02.272 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:02.267+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:02.430 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:02.425+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:02.450 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:02.445+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:02.718 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:07:02.711+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 40sec e200 v200) 2019-12-09T18:07:02.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:02.733+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:02.959 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:02.954+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:02.975 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:02.970+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:03.170 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:03.165+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:03.269 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:03.264+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:03.397 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:03.391+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:03.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:03.457+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:03.704 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:03.699+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:03.936 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:03.930+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:03.997 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:03.991+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:04.156 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:04.152+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:04.201 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:24:00.820 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.211 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:24:00.827 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.221 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:24:00.834 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.221 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:24:00.840 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.228 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:24:00.827 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.232 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:24:00.831 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.272 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:04.267+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:04.298 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:24:00.917 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.306 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:24:00.925 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.355 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:24:00.954 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:04.376 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:04.370+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:04.428 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:04.424+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:04.689 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:04.684+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:04.936 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:04.931+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:04.994 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:04.988+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:05.186 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:05.181+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:05.263 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:05.258+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:05.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:05.396+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:05.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:05.476+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:05.675 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:05.671+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:05.926 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:05.920+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:06.036 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:06.029+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:06.213 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:06.209+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:06.308 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:06.303+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:06.420 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:06.414+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:06.508 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:06.499+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:06.638 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:06.633+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:06.911 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:06.904+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:07.054 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:07.049+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:07.262 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:07.258+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:07.314 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:07.310+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:07.473 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:07.467+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:07.545 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:07.539+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:07.672 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:07.668+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:07.741 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:07:07.735+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 40sec e200 v200) 2019-12-09T18:07:07.929 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:07.924+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:08.049 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:08.042+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:08.246 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:08.241+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:08.328 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:08.318+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:08.438 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:08.432+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:08.565 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:08.559+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:08.701 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:08.696+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:08.893 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:08.887+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:09.007 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:09.002+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:09.278 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:09.271+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:09.384 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:09.376+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:09.408 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:09.403+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:09.573 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:09.567+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:09.755 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:09.750+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:09.885 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:09.880+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:10.053 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:10.047+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:10.237 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:10.232+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:10.388 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:10.383+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:10.444 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:10.437+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:10.589 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:10.583+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:10.801 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:10.790+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:10.888 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:10.883+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:11.040 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:11.033+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:11.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:11.214+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:11.400 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:11.394+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:11.404 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:11.399+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:11.577 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:11.571+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:11.835 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:11.829+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:11.924 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:07:11.915+0000 1694f700 -1 osd.2 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.126260+0000 front 2019-12-09T18:05:50.139537+0000 (oldest deadline 2019-12-09T18:06:30.758191+0000) 2019-12-09T18:07:12.081 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:07:12.074+0000 1694f700 -1 osd.0 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.956110+0000 front 2019-12-09T18:05:50.372877+0000 (oldest deadline 2019-12-09T18:06:30.346662+0000) 2019-12-09T18:07:12.190 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:12.185+0000 1694f700 -1 osd.5 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.893569+0000 front 2019-12-09T18:05:50.178772+0000 (oldest deadline 2019-12-09T18:06:33.363432+0000) 2019-12-09T18:07:12.404 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:07:12.398+0000 1694f700 -1 osd.1 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.976518+0000 front 2019-12-09T18:05:50.137335+0000 (oldest deadline 2019-12-09T18:06:30.042627+0000) 2019-12-09T18:07:12.453 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:07:12.444+0000 1694f700 -1 osd.6 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:49.914355+0000 front 2019-12-09T18:05:49.957151+0000 (oldest deadline 2019-12-09T18:06:30.499083+0000) 2019-12-09T18:07:12.540 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:12.533+0000 1694f700 -1 osd.4 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:50.176010+0000 front 2019-12-09T18:05:50.128206+0000 (oldest deadline 2019-12-09T18:06:31.996189+0000) 2019-12-09T18:07:12.803 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:07:12.789+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 40sec e200 v200) 2019-12-09T18:07:12.890 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:07:12.878+0000 1694f700 -1 osd.7 200 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:05:46.856769+0000 front 2019-12-09T18:05:47.721638+0000 (oldest deadline 2019-12-09T18:06:30.797308+0000) 2019-12-09T18:07:46.226 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:46.217+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:47.224 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:47.221+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:48.191 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:48.188+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:49.194 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:49.190+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:50.213 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:50.210+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:51.217 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:51.213+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:52.206 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:52.203+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:53.261 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:53.257+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:54.251 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:54.247+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:55.247 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:55.242+0000 1694f700 -1 osd.4 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:55.852 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:55.837+0000 3827c700 -1 osd.4 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.129 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:55.957+0000 3827c700 -1 osd.4 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.174 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:56.169+0000 3aa81700 -1 osd.5 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.180 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:56.174+0000 3aa81700 -1 osd.5 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.216 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:56.206+0000 3aa81700 -1 osd.5 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.351 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:56.045+0000 3827c700 -1 osd.4 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.357 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:56.065+0000 3927e700 -1 osd.4 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.553 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:56.465+0000 1694f700 -1 osd.4 204 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:56.637 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:56.495+0000 3827c700 -1 osd.5 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:56.699 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:07:56.562+0000 38a7d700 -1 osd.5 204 build_incremental_map_msg missing incremental map 204 2019-12-09T18:07:57.513 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:57.509+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:58.568 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:58.564+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:07:59.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:07:59.571+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:00.586 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:00.582+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:01.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:01.589+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:02.635 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:02.627+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:03.657 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:03.652+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:04.635 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:04.630+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:05.625 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:05.621+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:06.640 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:06.635+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:07.690 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:07.685+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:08.738 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:08.733+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:09.702 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:09.697+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:10.659 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:10.654+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:11.145 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:11.139+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:11.165 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:11.159+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:11.669 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:11.664+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:12.142 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:12.137+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:12.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:12.200+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:12.639 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:12.634+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:13.143 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:13.135+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:13.219 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:13.214+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:13.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:13.341+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:13.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:13.650+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:14.165 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:14.160+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:14.261 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:14.256+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:14.286 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:14.280+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:14.378 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:14.374+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:14.625 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:14.620+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:14.627 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:14.623+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:15.083 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:15.077+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:15.206 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:15.200+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:15.218 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:15.213+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:15.260 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:15.255+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:15.355 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:15.347+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:15.584 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:15.579+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:15.586 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:15.583+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:16.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:16.051+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:16.161 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:16.155+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:16.183 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:16.177+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:16.291 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:16.285+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:16.414 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:16.400+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:16.545 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:16.538+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:16.545 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:16.540+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:17.055 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:17.049+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:17.168 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:17.162+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:17.215 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:17.210+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:17.326 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:17.320+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:17.448 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:17.442+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:17.569 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:17.564+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:17.570 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:17.565+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:18.082 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:18.077+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:18.133 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:18.129+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:18.225 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:18.219+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:18.356 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:18.351+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:18.500 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:18.495+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:18.567 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:18.561+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:18.567 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:18.563+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:19.080 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:19.074+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:19.137 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:19.132+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:19.227 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:19.222+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:19.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:19.404+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:19.499 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:19.494+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:19.550 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:19.544+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:19.550 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:19.546+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:20.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:20.045+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:20.141 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:20.135+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:20.260 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:20.256+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:20.435 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:20.429+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:20.496 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:20.491+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:20.569 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:20.563+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:20.572 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:20.568+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:21.090 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:21.083+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:21.189 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:21.184+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:21.305 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:21.299+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:21.419 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:21.413+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:21.475 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:21.469+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:21.597 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:21.590+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:21.597 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:21.592+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:22.054 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:22.048+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:22.179 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:22.173+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:22.287 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:22.283+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:22.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:22.390+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:22.495 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:22.490+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:22.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:22.602+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:22.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:22.604+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:23.026 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:23.020+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:23.173 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:23.168+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:23.245 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:23.240+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:23.362 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:23.357+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:23.477 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:23.472+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:23.592 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:23.585+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:23.592 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:23.587+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:24.067 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:24.062+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:24.206 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:24.201+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:24.254 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:24.248+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:24.339 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:24.334+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:24.479 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:24.472+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:24.591 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:24.586+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:24.592 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:24.587+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:25.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:25.086+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:25.211 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:25.206+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:25.224 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:25.220+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:25.331 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:25.326+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:25.520 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:25.513+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:25.611 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:25.606+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:25.613 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:25.609+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:26.124 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:26.120+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:26.245 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:26.238+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:26.263 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:26.257+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:26.356 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:26.351+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:26.561 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:26.556+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:26.645 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:26.639+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:26.645 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:26.640+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:27.115 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:27.109+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:27.242 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:27.236+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:27.287 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:27.281+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:27.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:27.370+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:27.573 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:27.568+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:27.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:27.602+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:27.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:27.603+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:28.087 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:28.082+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:28.134 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:28.128+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:28.278 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:28.272+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:28.295 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:28.289+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:28.372 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:28.366+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:28.545 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:28.538+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:28.583 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:28.579+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:28.584 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:28.580+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:29.101 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:29.096+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:29.110 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:29.104+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:29.254 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:29.251+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:29.261 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:29.255+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:29.425 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:29.419+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:29.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:29.543+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:29.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:29.545+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:29.587 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:29.584+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:30.103 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:30.097+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:30.148 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:30.138+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:30.212 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:30.208+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:30.242 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:30.237+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:30.399 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:30.393+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:30.519 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:30.510+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:30.520 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:30.511+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:30.639 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:30.634+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:31.089 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:31.083+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:31.151 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:31.146+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:31.212 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:31.209+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:31.237 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:31.228+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:31.364 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:31.358+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:31.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:31.483+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:31.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:31.485+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:31.630 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:31.626+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:32.127 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:32.122+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:32.170 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:32.164+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:32.215 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:32.210+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:32.276 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:32.267+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:32.342 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:32.337+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:32.460 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:32.455+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:32.461 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:32.457+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:32.662 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:32.658+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:33.152 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:33.146+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:33.169 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:33.163+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:33.201 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:33.196+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:33.254 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:33.248+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:33.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:33.370+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:33.451 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:33.446+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:33.452 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:33.448+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:33.640 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:33.635+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:34.133 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:34.124+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:34.136 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:34.126+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:34.155 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:34.149+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:34.177 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:34.173+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:34.246 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:34.240+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:34.403 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:34.397+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:34.413 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:34.408+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:34.414 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:34.410+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:34.601 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:34.597+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:35.132 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:35.126+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:35.156 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:35.152+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:35.158 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:35.153+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:35.161 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:35.156+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:35.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:35.199+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:35.390 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:35.371+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:35.392 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:35.386+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:35.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:35.372+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:35.562 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:35.558+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:36.119 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:36.112+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:36.119 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:36.114+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:36.152 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:36.147+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:36.161 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:36.158+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:36.247 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:36.241+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:36.367 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:36.364+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:36.369 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:36.366+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:36.429 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:36.424+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:36.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:36.540+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:37.154 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:37.149+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:37.166 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:37.161+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:37.169 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:37.164+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:37.174 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:37.168+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:37.248 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:37.242+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:37.357 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:37.353+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:37.363 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:37.357+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:37.443 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:37.437+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:37.512 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:37.509+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:38.142 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:38.136+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:38.167 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:38.162+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:38.171 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:38.165+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:38.171 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:38.166+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:38.242 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:38.237+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:38.354 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:38.350+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:38.357 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:38.353+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:38.458 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:38.452+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:38.547 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:38.543+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:39.169 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:39.163+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:39.187 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:39.182+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:39.218 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:39.212+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:39.220 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:39.214+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:39.221 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:39.216+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:39.333 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:39.328+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:39.336 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:39.331+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:39.425 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:39.418+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:39.602 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:39.596+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:40.198 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:40.192+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:40.218 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:40.213+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:40.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:40.258+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:40.265 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:40.259+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:40.265 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:40.260+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:40.312 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:40.306+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:40.312 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:40.307+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:40.419 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:40.414+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:40.590 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:40.585+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:41.176 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:41.171+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:41.185 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:41.178+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:41.235 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:41.229+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:41.317 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:41.311+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:41.320 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:41.315+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:41.352 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:41.346+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:41.353 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:41.348+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:41.382 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:41.377+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:41.566 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:41.560+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:42.146 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:42.141+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:42.210 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:42.205+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:42.267 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:42.261+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:42.310 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:42.305+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:42.310 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:42.306+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:42.330 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:42.324+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:42.331 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:42.326+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:42.374 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:42.369+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:42.559 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:42.554+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:43.112 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:43.106+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:43.233 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:43.228+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:43.259 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:43.254+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:43.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:43.277+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:43.285 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:43.281+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:43.360 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:43.355+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:43.361 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:43.355+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:43.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:43.391+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:43.549 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:43.544+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:44.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:44.086+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:44.206 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:44.200+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:44.293 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:44.287+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:44.294 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:44.289+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:44.315 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:44.306+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:44.407 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:44.401+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:44.407 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:44.402+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:44.413 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:44.408+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:44.566 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:44.561+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:45.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:45.098+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:45.242 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:45.236+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:45.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:45.254+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:45.261 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:45.256+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:45.337 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:45.331+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:45.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:45.373+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:45.405 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:45.399+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:45.406 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:45.401+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:45.574 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:45.568+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:46.101 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:46.097+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:46.222 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:46.216+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:46.223 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:46.218+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:46.264 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:46.260+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:46.359 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:46.355+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:46.366 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:46.361+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:46.383 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:46.378+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:46.383 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:46.380+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:46.383 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:08:46.353+0000 20e25700 -1 RGWWatcher::handle_error cookie 5399339296 err (107) Transport endpoint is not connected 2019-12-09T18:08:46.566 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:46.561+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:47.091 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:47.085+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:47.259 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:47.253+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:47.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:47.255+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:47.279 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:47.274+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:47.327 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:47.322+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:47.350 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:47.345+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:47.350 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:47.346+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:47.398 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:47.393+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:47.608 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:47.603+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:48.122 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:48.117+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:48.247 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:08:48.241+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e205 v205) 2019-12-09T18:08:48.247 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:48.241+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:48.248 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:48.243+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:48.285 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:48.280+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:48.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:48.317+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:48.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:48.317+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:48.334 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:48.329+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:48.399 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:48.394+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:48.577 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:48.572+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:49.149 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:49.144+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:49.213 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:49.208+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:49.216 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:49.212+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:49.243 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:49.237+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:49.284 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:49.278+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:49.284 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:49.280+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:49.298 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:49.293+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:49.448 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:49.443+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:49.550 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:49.544+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:50.126 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:50.121+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:50.220 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:50.212+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:50.220 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:50.215+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:50.237 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:50.232+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:50.317 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:50.312+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:50.318 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:50.313+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:50.319 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:50.314+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:50.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:50.410+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:50.582 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:50.576+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:51.164 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:51.159+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:51.232 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:51.228+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:51.261 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:51.255+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:51.262 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:51.257+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:51.281 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:51.277+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:51.324 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:51.319+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:51.335 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:51.323+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:51.416 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:51.408+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:51.562 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:51.557+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:52.152 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:52.147+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:52.206 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:52.202+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:52.223 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:52.214+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:52.227 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:52.216+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:52.258 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:52.249+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:52.334 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:52.328+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:52.334 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:52.330+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:52.390 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:52.385+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:52.554 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:52.542+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:53.191 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:53.186+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:53.205 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:53.198+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:53.209 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:53.204+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:53.262 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:08:53.258+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e205 v205) 2019-12-09T18:08:53.262 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:53.257+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:53.266 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:53.261+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:53.341 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:53.336+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:53.341 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:53.338+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:53.434 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:53.424+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:53.518 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:53.513+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:54.211 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:54.206+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:54.230 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:54.225+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:54.239 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:54.234+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:54.256 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:54.250+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:54.257 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:54.252+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:54.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:54.307+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:54.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:54.309+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:54.407 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:54.404+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:54.551 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:54.545+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:55.203 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:55.198+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:55.216 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:55.211+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:55.258 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:55.253+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:55.278 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:55.272+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:55.279 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:55.274+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:55.346 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:55.340+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:55.347 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:55.342+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:55.408 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:55.403+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:55.550 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:55.544+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:56.166 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:56.162+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:56.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:56.200+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:56.268 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:56.262+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:56.269 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:56.265+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:56.292 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:56.287+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:56.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:56.317+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:56.324 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:56.318+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:56.438 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:56.433+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:56.549 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:56.544+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:57.199 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:57.195+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:57.245 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:57.240+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:57.259 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:57.254+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:57.290 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:57.282+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:57.291 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:57.285+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:57.374 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:57.368+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:57.374 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:57.370+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:57.435 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:57.431+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:57.597 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:57.591+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:58.161 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:58.156+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:58.231 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:58.227+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:58.255 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:58.250+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:58.278 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:58.273+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:58.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:58.275+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:58.309 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:08:58.304+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e205 v205) 2019-12-09T18:08:58.377 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:58.370+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:58.378 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:58.372+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:58.450 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:58.445+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:58.571 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:58.565+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:08:59.168 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:08:59.163+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:08:59.229 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:08:59.224+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:08:59.273 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:08:59.267+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:08:59.331 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:59.325+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:08:59.332 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:08:59.327+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:08:59.364 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:59.360+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:08:59.365 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:08:59.361+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:08:59.484 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:08:59.479+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:08:59.607 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:08:59.601+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:00.198 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:00.195+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:00.245 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:00.236+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:00.254 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:00.245+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:00.330 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:00.325+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:00.331 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:00.326+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:00.336 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:00.332+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:00.336 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:00.333+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:00.445 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:00.441+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:00.621 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:00.616+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:01.233 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:01.229+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:01.251 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:01.246+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:01.291 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:01.287+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:01.292 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:01.288+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:01.293 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:01.287+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:01.294 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:01.289+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:01.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:01.292+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:01.423 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:01.418+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:01.668 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:01.663+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:02.239 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:02.236+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:02.269 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:02.264+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:02.272 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:02.267+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:02.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:02.269+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:02.315 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:02.311+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:02.326 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:02.322+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:02.327 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:02.323+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:02.413 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:02.408+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:02.711 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:02.704+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:03.235 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:03.231+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:03.288 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:03.281+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:03.289 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:03.284+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:03.305 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:03.300+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:03.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:03.309+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:03.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:03.310+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:03.318 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:03.315+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:03.320 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:09:03.315+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e205 v205) 2019-12-09T18:09:03.395 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:03.391+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:03.679 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:03.673+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:04.268 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:04.263+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:04.271 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:04.267+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:04.280 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:04.275+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:04.284 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:04.279+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:04.335 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:04.322+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:04.336 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:04.331+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:04.341 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:04.336+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:04.427 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:04.422+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:04.709 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:04.703+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:05.271 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:05.262+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:05.272 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:05.263+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:05.278 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:05.274+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:05.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:05.307+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:05.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:05.309+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:05.338 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:05.331+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:05.369 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:05.361+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:05.477 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:05.461+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:05.705 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:05.696+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:06.259 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:06.255+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:06.260 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:06.257+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:06.291 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:06.284+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:06.291 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:06.286+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:06.319 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:06.315+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:06.362 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:06.357+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:06.394 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:06.388+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:06.468 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:06.462+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:06.682 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:06.677+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:07.294 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:07.279+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:07.294 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:07.281+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:07.309 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:07.304+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:07.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:07.308+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:07.316 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:07.312+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:07.343 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:07.337+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:07.437 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:07.429+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:07.514 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:09:07.510+0000 1694f700 -1 osd.0 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244417+0000 front 2019-12-09T18:07:30.273550+0000 (oldest deadline 2019-12-09T18:08:14.132836+0000) 2019-12-09T18:09:07.681 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:07.675+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:08.246 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:08.236+0000 1694f700 -1 osd.4 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:31.091171+0000 front 2019-12-09T18:07:29.324344+0000 (oldest deadline 2019-12-09T18:08:13.997727+0000) 2019-12-09T18:09:08.246 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:08.239+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e202) 2019-12-09T18:09:08.328 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:09:08.324+0000 1694f700 -1 osd.2 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.312344+0000 front 2019-12-09T18:07:30.109701+0000 (oldest deadline 2019-12-09T18:08:14.234224+0000) 2019-12-09T18:09:08.334 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:08.330+0000 1694f700 -1 osd.3 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:07:53.352826+0000 (oldest deadline 2019-12-09T18:08:33.352826+0000) 2019-12-09T18:09:08.338 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:08.334+0000 1694f700 -1 osd.3 205 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:08.364 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:09:08.359+0000 1694f700 -1 osd.1 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.938332+0000 front 2019-12-09T18:07:30.979335+0000 (oldest deadline 2019-12-09T18:08:10.235106+0000) 2019-12-09T18:09:08.394 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:09:08.388+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e205 v205) 2019-12-09T18:09:08.483 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:09:08.477+0000 1694f700 -1 osd.7 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:30.244395+0000 front 2019-12-09T18:07:30.241772+0000 (oldest deadline 2019-12-09T18:08:10.864222+0000) 2019-12-09T18:09:08.668 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:09:08.663+0000 1694f700 -1 osd.5 205 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:07:27.209180+0000 front 2019-12-09T18:07:27.598726+0000 (oldest deadline 2019-12-09T18:08:12.439447+0000) 2019-12-09T18:09:09.564 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:09:09.466+0000 1694f700 -1 osd.3 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e204) 2019-12-09T18:09:09.570 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:09.557+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202) 2019-12-09T18:09:10.555 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:10.551+0000 1694f700 -1 osd.4 207 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:119645 2.6s0 2:632b6490:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_kappa%2fe5c272a994334e72ac27171df27a463c:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202) 2019-12-09T18:09:28.998 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:09:28.989+0000 3aa81700 -1 osd.4 208 build_incremental_map_msg missing incremental map 204 2019-12-09T18:09:53.871 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:26:50.490 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:53.878 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:26:50.498 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:53.958 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:26:50.557 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:53.963 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:26:50.562 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:53.992 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:26:50.612 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:54.029 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:26:50.648 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:54.095 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:26:50.714 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:54.270 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:26:50.868 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:09:54.277 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:26:50.897 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:10:57.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:10:57.472+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:10:58.037 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:10:58.030+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:10:58.354 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:10:58.349+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:10:58.498 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:10:58.492+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:10:58.575 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:10:58.570+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:10:59.017 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:10:59.010+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:10:59.325 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:10:59.319+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:10:59.329 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:10:59.324+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:10:59.543 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:10:59.536+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:10:59.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:10:59.619+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:10:59.764 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:10:59.756+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:00.036 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:00.031+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:00.103 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:00.097+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:00.337 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:00.328+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:00.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:00.361+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:00.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:00.508+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:00.620 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:00.613+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:00.721 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:00.715+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:01.036 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:01.029+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:01.128 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:01.122+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:01.340 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:01.334+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:01.343 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:01.338+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:01.402 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:27:58.021 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.412 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:27:58.031 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.423 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:27:58.042 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.437 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:27:58.046 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.498 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:27:58.117 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.503 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:27:58.122 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.513 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:27:58.111 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.523 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:27:58.122 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.552 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:01.546+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:01.632 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:27:58.231 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:11:01.662 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:01.658+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:01.761 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:01.755+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:02.081 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:02.075+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:02.154 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:02.149+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:02.295 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:02.289+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:02.371 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:02.366+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:02.565 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:02.560+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:02.662 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:02.657+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:02.746 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:02.740+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:03.102 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:03.095+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:03.202 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:03.196+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:03.277 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:03.272+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:03.345 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:03.339+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:03.601 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:03.595+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:03.646 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:03.641+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:03.797 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:03.790+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:04.156 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:04.149+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:04.230 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:04.222+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:04.262 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:04.255+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:04.326 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:04.320+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:04.644 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:04.638+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:04.648 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:04.641+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:04.784 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:04.778+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:05.188 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:05.182+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:05.199 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:05.193+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:05.314 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:05.307+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:05.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:05.361+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:05.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:05.627+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:05.688 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:05.679+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:05.768 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:05.762+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:06.168 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:06.163+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:06.218 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:06.212+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:06.331 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:06.326+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:06.340 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:06.334+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:06.615 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:06.608+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:06.734 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:06.728+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:06.796 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:06.790+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:07.215 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:07.209+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:07.245 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:07.239+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:07.296 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:07.289+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:07.296 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:07.290+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:07.675 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:07.659+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:07.769 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:07.762+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:07.795 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:07.788+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:08.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:08.207+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:08.265 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:08.259+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:08.296 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:08.290+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:08.347 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:08.340+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:08.713 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:08.707+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:08.751 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:08.746+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:08.785 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:08.779+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:09.192 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:09.187+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:09.312 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:09.306+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:09.318 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:09.313+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:09.339 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:09.334+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:09.692 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:09.686+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:09.773 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:09.767+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:09.798 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:09.792+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:10.240 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:10.235+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:10.306 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:10.301+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:10.315 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:10.309+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:10.340 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:10.335+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:10.733 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:10.727+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:10.814 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:10.807+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:10.815 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:10.809+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:11.224 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:11.218+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:11.297 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:11.292+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:11.361 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:11.355+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:11.393 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:11.388+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:11.733 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:11.727+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:11.837 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:11.832+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:11.839 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:11.833+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:12.269 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:12.264+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:12.270 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:12.264+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:12.385 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:12.378+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:12.425 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:12.419+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:12.738 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:12.732+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:12.845 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:12.836+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:12.847 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:12.842+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:13.245 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:13.239+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:13.247 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:13.242+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:13.417 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:13.405+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:13.474 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:13.468+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:13.771 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:13.764+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:13.832 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:13.826+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:13.832 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:13.825+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:14.222 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:14.210+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:14.229 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:14.223+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:14.374 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:14.369+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:14.502 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:14.496+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:14.779 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:14.774+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:14.792 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:14.786+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:14.861 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:14.855+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:15.216 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:15.211+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:15.232 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:15.226+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:15.400 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:15.395+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:15.568 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:15.562+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:15.775 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:15.768+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:15.790 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:15.784+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:15.868 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:15.862+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:16.231 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:16.225+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:16.243 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:16.238+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:16.384 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:16.377+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:16.617 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:16.611+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:16.771 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:16.765+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:16.804 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:16.798+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:16.905 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:16.900+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:17.249 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:17.243+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:17.268 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:17.262+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:17.376 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:17.369+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:17.636 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:17.630+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:17.729 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:17.723+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:17.863 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:17.857+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:17.865 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:17.859+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:18.205 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:18.199+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:18.270 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:18.264+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:18.333 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:18.328+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:18.618 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:18.613+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:18.700 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:18.694+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:18.819 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:18.814+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:18.870 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:18.863+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:19.215 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:19.210+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:19.233 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:19.227+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:19.330 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:19.324+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:19.626 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:19.619+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:19.727 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:19.721+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:19.843 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:19.837+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:19.858 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:19.852+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:20.199 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:20.192+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:20.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:20.244+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:20.373 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:20.368+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:20.605 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:20.600+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:20.717 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:20.712+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:20.808 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:20.802+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:20.853 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:20.846+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:21.237 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:21.231+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:21.256 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:21.251+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:21.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:21.363+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:21.657 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:21.651+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:21.730 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:21.725+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:21.776 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:21.770+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:21.886 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:21.880+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:22.233 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:22.228+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:22.248 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:22.242+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:22.345 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:22.341+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:22.707 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:22.700+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:22.738 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:22.732+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:22.780 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:22.775+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:22.912 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:22.905+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:23.264 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:23.258+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:23.270 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:23.264+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:23.377 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:23.371+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:23.730 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:23.724+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:23.750 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:23.746+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:23.753 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:23.748+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:23.922 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:23.916+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:24.229 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:24.221+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:24.258 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:24.253+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:24.392 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:24.388+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:24.690 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:24.684+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:24.729 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:24.724+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:24.808 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:24.802+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:24.968 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:24.963+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:25.183 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:25.177+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:25.285 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:25.280+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:25.395 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:25.388+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:25.658 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:25.653+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:25.739 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:25.734+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:25.809 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:25.805+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:26.020 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:26.012+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:26.184 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:26.177+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:26.279 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:26.274+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:26.410 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:26.405+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:26.639 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:26.633+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:26.712 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:26.708+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:26.831 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:26.825+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:27.014 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:27.009+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:27.203 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:27.198+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:27.298 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:27.293+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:27.412 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:27.406+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:27.650 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:27.645+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:27.721 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:27.717+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:27.844 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:27.839+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:28.006 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:28.002+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:28.184 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:28.179+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:28.265 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:28.260+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:28.393 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:28.389+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:28.629 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:28.625+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:28.719 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:28.715+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:28.882 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:28.878+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:28.991 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:28.986+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:29.106 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:11:29.101+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e211 v211) 2019-12-09T18:11:29.184 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:29.180+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:29.282 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:29.277+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:29.434 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:29.428+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:29.627 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:29.623+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:29.736 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:29.730+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:29.875 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:29.870+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:29.948 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:29.943+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:30.229 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:30.224+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:30.321 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:30.317+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:30.391 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:30.386+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:30.647 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:30.642+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:30.742 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:30.735+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:30.895 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:30.890+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:30.970 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:30.966+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:31.200 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:31.196+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:31.299 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:31.295+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:31.443 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:31.438+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:31.653 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:31.650+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:31.711 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:31.706+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:31.932 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:31.927+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:31.958 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:31.955+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:32.159 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:32.155+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:32.344 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:32.339+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:32.412 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:32.406+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:32.690 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:32.685+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:32.705 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:32.700+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:32.949 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:32.944+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:32.970 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:32.965+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:33.178 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:33.174+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:33.368 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:33.362+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:33.428 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:33.422+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:33.683 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:33.678+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:33.724 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:33.719+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:33.966 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:33.962+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:33.991 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:33.986+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:34.159 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:11:34.156+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e211 v211) 2019-12-09T18:11:34.205 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:34.200+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:34.372 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:34.368+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:34.422 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:34.417+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:34.660 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:34.654+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:34.723 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:34.718+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:34.954 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:34.949+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:34.978 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:34.973+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:35.232 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:35.227+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:35.388 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:35.383+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:35.403 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:35.398+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:35.660 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:35.655+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:35.761 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:35.756+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:35.921 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:35.917+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:36.009 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:36.004+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:36.232 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:36.226+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:36.403 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:36.399+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:36.447 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:36.437+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:36.660 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:36.654+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:36.741 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:36.736+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:36.968 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:36.962+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:36.994 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:36.988+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:37.279 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:37.273+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:37.372 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:37.364+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:37.413 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:37.407+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:37.661 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:37.654+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:37.741 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:37.735+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:37.981 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:37.975+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:38.004 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:37.998+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:38.283 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:38.277+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:11:38.385 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:11:38.380+0000 1694f700 -1 osd.0 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.458421+0000 front 2019-12-09T18:10:15.497860+0000 (oldest deadline 2019-12-09T18:10:59.524942+0000) 2019-12-09T18:11:38.388 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:11:38.383+0000 1694f700 -1 osd.1 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:14.482644+0000 front 2019-12-09T18:10:14.251150+0000 (oldest deadline 2019-12-09T18:10:58.321511+0000) 2019-12-09T18:11:38.656 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:11:38.650+0000 1694f700 -1 osd.4 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.217067+0000 front 2019-12-09T18:10:15.182088+0000 (oldest deadline 2019-12-09T18:10:57.429254+0000) 2019-12-09T18:11:38.760 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:11:38.754+0000 1694f700 -1 osd.2 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:15.693257+0000 front 2019-12-09T18:10:15.435017+0000 (oldest deadline 2019-12-09T18:10:58.806573+0000) 2019-12-09T18:11:38.945 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:11:38.938+0000 1694f700 -1 osd.5 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:16.254079+0000 front 2019-12-09T18:10:16.210596+0000 (oldest deadline 2019-12-09T18:10:57.876769+0000) 2019-12-09T18:11:39.014 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:11:39.008+0000 1694f700 -1 osd.7 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746573+0000 front 2019-12-09T18:10:13.730511+0000 (oldest deadline 2019-12-09T18:10:58.541646+0000) 2019-12-09T18:11:39.244 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:11:39.238+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e211 v211) 2019-12-09T18:11:39.253 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:11:39.248+0000 1694f700 -1 osd.6 211 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:10:13.746387+0000 front 2019-12-09T18:10:13.930599+0000 (oldest deadline 2019-12-09T18:10:57.345835+0000) 2019-12-09T18:12:18.488 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:18.471+0000 3827c700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:18.795 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:18.747+0000 33a73700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:18.824 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:18.782+0000 3927e700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:19.030 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:18.814+0000 33a73700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:19.086 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:18.892+0000 3927e700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:19.090 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:18.902+0000 37a7b700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:19.390 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:19.188+0000 3827c700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:19.469 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:19.226+0000 37a7b700 -1 osd.3 216 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:22.607 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:22.586+0000 3aa81700 -1 osd.3 217 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:23.762 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:23.737+0000 3b282700 -1 osd.3 217 build_incremental_map_msg missing incremental map 216 2019-12-09T18:12:30.822 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:30.816+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:31.326 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:31.322+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:31.864 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:31.859+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:32.364 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:32.355+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:32.825 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:32.820+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:32.964 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:32.958+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:33.330 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:33.324+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:33.809 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:33.803+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:33.836 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:33.830+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:33.932 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:33.926+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:34.052 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:34.047+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:34.343 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:34.338+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:34.844 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:34.839+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:34.862 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:34.857+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:34.971 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:34.961+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:35.085 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:35.076+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:35.327 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:35.322+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:35.341 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:35.325+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:35.821 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:35.816+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:35.860 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:35.855+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:35.949 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:35.943+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:36.105 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:36.098+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:36.347 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:36.342+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:36.350 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:36.346+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:36.839 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:36.833+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:36.845 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:36.840+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:36.933 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:36.927+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:37.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:37.094+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:37.307 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:37.302+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:37.380 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:37.374+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:37.811 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:37.805+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:37.848 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:37.843+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:37.901 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:37.895+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:38.065 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:38.060+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:38.287 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:38.281+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:38.372 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:38.367+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:38.837 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:38.832+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:38.892 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:38.886+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:38.904 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:38.898+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:39.046 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:39.042+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:39.307 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:39.301+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:39.422 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:39.415+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:39.798 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:39.792+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:39.895 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:39.891+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:39.928 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:39.923+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:40.064 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:40.060+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:40.358 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:40.352+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:40.464 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:40.459+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:40.806 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:40.802+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:40.930 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:40.924+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:40.947 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:40.942+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:41.067 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:41.061+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:41.351 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:41.345+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:41.436 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:41.431+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:41.855 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:41.849+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:41.919 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:41.914+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:41.995 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:41.990+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:42.056 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:42.050+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:42.387 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:42.382+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:42.449 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:42.444+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:42.870 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:42.854+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:42.959 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:42.953+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:43.042 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:43.037+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:43.070 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:43.064+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:43.433 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:43.427+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:43.433 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:43.428+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:43.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:43.885+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:44.001 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:43.995+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:44.059 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:44.054+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:44.091 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:44.086+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:44.407 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:44.402+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:44.452 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:44.446+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:44.927 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:44.923+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:44.998 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:44.993+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:45.059 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:45.054+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:45.129 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:45.123+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:45.455 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:45.450+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:45.457 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:45.452+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:45.974 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:45.969+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:46.029 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:46.010+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:46.038 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:46.032+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:46.168 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:46.163+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:46.432 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:46.426+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:46.443 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:46.438+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:46.987 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:46.982+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:47.002 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:46.998+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:47.053 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:47.037+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:47.207 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:47.202+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:47.420 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:47.414+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:47.478 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:47.474+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:47.992 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:47.986+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:48.026 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:48.022+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:48.028 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:48.023+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:48.034 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:48.029+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:48.259 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:48.254+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:48.379 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:48.373+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:48.439 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:48.434+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:49.020 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:49.016+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:49.023 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:49.019+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:49.085 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:49.080+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:49.086 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:49.081+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:49.312 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:49.307+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:49.370 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:49.365+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:49.458 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:49.452+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:49.987 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:49.983+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:50.045 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:50.040+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:50.045 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:50.041+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:50.097 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:50.093+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:50.341 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:50.334+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:50.360 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:50.355+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:50.507 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:50.502+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:50.956 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:50.951+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:51.012 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:51.006+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:51.065 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:51.060+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:51.073 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:51.068+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:51.373 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:51.369+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:51.394 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:51.388+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:51.508 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:51.503+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:51.990 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:51.986+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:52.023 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:52.018+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:52.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:52.037+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:52.087 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:52.080+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:52.362 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:52.358+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:52.437 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:52.432+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:52.483 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:52.477+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:52.970 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:52.965+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:53.043 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:53.037+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:53.094 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:53.090+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:53.118 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:53.113+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:53.404 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:53.400+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:53.472 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:53.461+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:53.484 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:53.476+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:53.993 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:53.988+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:54.064 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:54.059+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:54.076 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:54.071+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:54.102 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:54.098+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:54.442 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:54.436+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:54.461 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:54.456+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:54.496 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:54.491+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:55.034 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:55.029+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:55.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:55.087+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:55.108 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:55.103+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:55.123 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:55.118+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:12:55.124 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:55.120+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:55.433 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:55.428+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:55.476 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:55.470+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:55.520 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:55.514+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:56.033 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:56.028+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:56.070 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:56.065+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:56.097 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:56.092+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:12:56.100 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:56.096+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:56.103 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:56.097+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:56.438 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:56.433+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:56.511 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:56.507+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:56.516 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:56.512+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:57.061 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:57.056+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:57.098 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:57.094+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:57.121 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:57.117+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:57.154 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:57.148+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:12:57.154 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:57.150+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:57.446 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:57.441+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:57.474 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:57.470+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:57.505 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:57.500+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:58.109 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:58.103+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:58.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:58.110+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:58.121 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:58.117+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:58.159 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:58.153+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:12:58.159 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:58.155+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:58.438 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:58.435+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:58.485 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:58.482+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:58.522 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:58.519+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:12:59.099 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:12:59.095+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:12:59.144 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:59.139+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:12:59.146 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:12:59.143+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:12:59.148 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:12:59.144+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:12:59.172 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:12:59.168+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:12:59.443 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:12:59.439+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:12:59.452 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:12:59.447+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:12:59.503 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:12:59.498+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:00.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:00.110+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:00.143 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:00.138+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:00.145 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:00.140+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:00.145 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:00.141+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:00.176 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:00.171+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:00.416 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:00.411+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:00.492 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:00.487+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:00.494 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:00.490+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:01.098 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:01.093+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:01.106 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:01.100+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:01.167 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:01.163+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:01.170 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:01.164+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:01.171 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:01.166+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:01.451 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:01.447+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:01.482 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:01.479+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:01.538 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:01.535+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:02.111 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:02.106+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:02.124 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:02.120+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:02.151 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:02.146+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:02.165 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:02.160+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:02.166 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:02.162+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:02.474 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:02.469+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:02.482 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:02.478+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:02.524 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:02.519+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:03.123 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:03.118+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:03.123 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:03.117+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:03.191 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:03.185+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:03.208 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:03.202+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:03.209 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:03.204+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:03.462 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:03.458+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:03.528 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:03.523+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:03.529 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:03.524+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:04.160 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:04.155+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:04.170 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:04.165+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:04.171 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:04.166+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:04.172 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:04.168+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:04.200 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:04.196+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:04.494 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:04.490+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:04.496 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:04.492+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:04.511 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:04.507+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:04.578 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:13:04.573+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e217 v217) 2019-12-09T18:13:05.160 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:05.152+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:05.177 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:05.172+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:05.188 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:05.184+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:05.191 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:05.187+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:05.245 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:05.241+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:05.464 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:05.459+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:05.539 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:05.536+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:05.544 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:05.541+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:06.139 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:06.135+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:06.140 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:06.133+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:06.223 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:06.216+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:06.224 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:06.218+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:06.278 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:06.272+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:06.445 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:06.439+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:06.511 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:06.506+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:06.569 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:06.564+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:07.116 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:07.107+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:07.123 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:07.118+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:07.244 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:07.238+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:07.245 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:07.240+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:07.295 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:07.290+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:07.431 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:07.426+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:07.511 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:07.506+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:07.549 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:07.544+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:08.091 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:08.085+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:08.144 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:08.138+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:08.224 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:08.217+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:08.224 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:08.219+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:08.317 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:08.311+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:08.407 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:08.403+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:08.548 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:08.542+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:08.576 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:08.570+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:09.111 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:09.105+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:09.112 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:09.106+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:09.114 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:09.107+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:09.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:09.241+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:09.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:09.243+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:09.339 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:09.334+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:09.404 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:09.398+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:09.558 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:09.552+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:09.603 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:13:09.595+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e217 v217) 2019-12-09T18:13:09.618 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:09.612+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:10.091 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:10.085+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:10.156 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:10.143+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:10.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:10.145+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:10.285 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:10.280+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:10.288 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:10.283+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:10.310 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:10.304+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:10.350 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:13:10.322+0000 20e25700 -1 RGWWatcher::handle_error cookie 5394733968 err (107) Transport endpoint is not connected 2019-12-09T18:13:10.457 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:10.452+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:10.574 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:10.568+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:10.596 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:10.592+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:11.128 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:11.122+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:11.199 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:11.194+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:11.202 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:11.198+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:11.276 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:11.271+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:11.279 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:11.274+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:11.349 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:11.343+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:11.477 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:11.471+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:11.550 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:11.542+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:11.633 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:11.627+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:12.164 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:12.158+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:12.236 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:12.231+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:12.236 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:12.232+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:12.271 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:12.265+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:12.272 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:12.266+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:12.382 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:12.375+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:12.483 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:12.476+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:12.554 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:12.548+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:12.591 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:12.586+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:13.174 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:13.168+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:13.238 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:13.233+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:13.239 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:13.234+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:13.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:13.313+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:13.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:13.314+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:13.371 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:13.360+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:13.485 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:13.479+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:13.583 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:13.578+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:13.606 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:13.601+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:14.181 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:14.175+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:14.278 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:14.272+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:14.281 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:14.276+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:14.289 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:14.284+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:14.292 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:14.288+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:14.358 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:14.352+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:14.447 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:14.441+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:14.622 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:13:14.613+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e217 v217) 2019-12-09T18:13:14.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:14.619+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:14.630 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:14.625+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:15.138 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:15.133+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:15.240 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:15.235+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:15.243 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:15.238+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:15.327 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:15.322+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:15.330 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:15.326+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:15.331 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:15.326+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:15.434 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:15.430+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:15.632 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:15.627+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:15.665 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:15.659+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:16.147 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:16.142+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:16.199 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:16.192+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:16.200 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:16.194+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:16.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:16.343+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:16.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:16.344+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:16.373 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:16.366+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:16.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:16.444+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:16.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:16.619+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:16.686 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:16.680+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:17.172 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:17.166+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:17.241 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:17.236+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:17.244 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:17.239+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:17.352 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:17.346+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:17.354 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:17.348+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:17.374 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:17.368+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:17.485 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:17.476+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:17.659 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:17.653+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:17.717 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:17.708+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:18.145 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:18.139+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:18.257 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:18.251+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:18.260 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:18.255+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:18.338 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:18.333+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:18.339 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:18.334+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:18.362 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:18.355+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:18.504 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:18.498+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:18.707 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:18.701+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:18.754 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:18.748+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:19.163 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:19.157+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:19.273 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:19.267+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:19.276 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:19.271+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:19.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:19.341+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:19.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:19.344+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:19.397 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:19.391+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:19.482 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:19.477+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:19.644 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:13:19.635+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e217 v217) 2019-12-09T18:13:19.722 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:19.717+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:19.749 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:19.743+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:20.161 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:20.155+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:20.302 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:20.296+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:20.303 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:20.298+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:20.368 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:20.363+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:20.400 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:20.394+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:20.401 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:20.396+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:20.447 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:20.440+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:20.752 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:20.747+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:20.776 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:20.771+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:21.150 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:21.144+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:21.326 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:21.320+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:21.328 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:21.324+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:21.362 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:21.356+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:21.364 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:21.360+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:21.371 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:21.366+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:21.443 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:21.437+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:21.806 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:21.801+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:21.817 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:21.812+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:22.122 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:22.117+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:22.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:22.329+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:22.334 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:22.327+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:22.335 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:22.331+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:22.338 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:22.332+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:22.406 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:22.400+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:22.431 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:22.425+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:22.790 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:22.785+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:22.854 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:22.848+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:23.159 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:23.153+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:23.300 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:23.294+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:23.300 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:23.295+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:23.359 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:23.354+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:23.362 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:23.357+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:23.416 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:23.410+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:23.449 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:23.443+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:23.764 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:23.759+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:23.898 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:23.893+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:24.167 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:24.160+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:24.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:24.310+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:24.317 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:24.313+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:24.374 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:24.369+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:24.375 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:24.370+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:24.416 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:24.411+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:24.430 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:24.425+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:24.667 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:13:24.657+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e217 v217) 2019-12-09T18:13:24.766 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:24.761+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:24.940 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:24.935+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:25.175 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:25.169+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:25.333 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:25.328+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:25.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:25.330+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:25.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:25.344+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:25.351 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:25.346+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:25.415 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:25.409+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:25.437 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:25.431+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:25.753 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:25.748+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:25.992 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:25.987+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:26.185 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:26.179+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:26.298 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:26.293+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:26.299 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:26.294+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:26.382 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:26.375+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:26.382 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:26.376+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:26.419 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:26.414+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:26.446 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:26.440+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:26.801 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:26.795+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:27.014 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:27.008+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:27.151 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:27.145+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:27.331 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:27.325+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:27.332 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:27.327+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:27.431 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:27.424+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:27.432 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:27.426+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:27.433 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:27.428+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:27.465 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:27.454+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:27.840 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:27.835+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:28.023 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:28.017+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:28.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:28.127+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:28.313 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:28.307+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:28.313 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:28.309+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:28.403 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:28.394+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:28.404 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:28.396+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:28.452 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:28.447+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:28.464 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:28.459+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:28.848 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:28.842+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:29.042 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:13:29.037+0000 1694f700 -1 osd.5 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.882684+0000 front 2019-12-09T18:11:51.518727+0000 (oldest deadline 2019-12-09T18:12:34.566125+0000) 2019-12-09T18:13:29.136 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:13:29.130+0000 1694f700 -1 osd.0 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:53.112920+0000 front 2019-12-09T18:11:52.454199+0000 (oldest deadline 2019-12-09T18:12:32.790157+0000) 2019-12-09T18:13:29.301 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:29.294+0000 1694f700 -1 osd.4 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:46.299293+0000 front 2019-12-09T18:11:56.885028+0000 (oldest deadline 2019-12-09T18:12:30.533098+0000) 2019-12-09T18:13:29.301 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:29.296+0000 1694f700 -1 osd.4 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:29.414 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:29.408+0000 1694f700 -1 osd.3 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:12:14.663603+0000 (oldest deadline 2019-12-09T18:12:54.663603+0000) 2019-12-09T18:13:29.419 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:29.415+0000 1694f700 -1 osd.3 217 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:29.455 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:13:29.449+0000 1694f700 -1 osd.2 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:56.677807+0000 front 2019-12-09T18:11:49.398209+0000 (oldest deadline 2019-12-09T18:12:33.475036+0000) 2019-12-09T18:13:29.456 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:13:29.450+0000 1694f700 -1 osd.1 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:48.472228+0000 front 2019-12-09T18:11:48.629111+0000 (oldest deadline 2019-12-09T18:12:33.110168+0000) 2019-12-09T18:13:29.741 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:13:29.734+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e217 v217) 2019-12-09T18:13:29.815 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:13:29.810+0000 1694f700 -1 osd.7 217 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:11:49.505039+0000 front 2019-12-09T18:11:51.531889+0000 (oldest deadline 2019-12-09T18:12:31.173094+0000) 2019-12-09T18:13:30.511 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:30.425+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:13:30.814 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:13:30.723+0000 1694f700 -1 osd.3 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e215) 2019-12-09T18:13:31.442 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:13:31.431+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:120174 5.3 5:c609908c:::notify.5:head [watch ping cookie 5394733968 gen 1] snapc 0=[] ondisk+write+known_if_redirected e217) 2019-12-09T18:14:10.766 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:14:10.759+0000 34a75700 -1 osd.3 223 build_incremental_map_msg missing incremental map 192 2019-12-09T18:14:10.766 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:14:10.760+0000 34a75700 -1 osd.3 223 build_incremental_map_msg missing incremental map 193 2019-12-09T18:14:10.775 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:14:10.771+0000 34a75700 -1 osd.3 223 build_incremental_map_msg missing incremental map 216 2019-12-09T18:14:35.168 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:31:31.787 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.205 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:31:31.825 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.237 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:31:31.857 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.243 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:31:31.863 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.257 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:31:31.856 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.268 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:31:31.867 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.399 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:31:32.019 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.462 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:31:32.081 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:35.506 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:31:32.105 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:14:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:testPrefixAndLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T18:14:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:testTooLongName (test.functional.tests.TestContainer) ... ok 2019-12-09T18:15:00.469 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:31:57.088 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.474 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:31:57.093 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.481 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:31:57.101 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.490 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:31:57.109 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.501 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:31:57.099 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.515 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:31:57.114 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.516 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:31:57.135 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.534 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:31:57.153 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:00.601 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:31:57.200 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:15:24.784 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:24.776+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:24.980 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:24.974+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:25.737 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:25.731+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:25.795 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:25.788+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:25.976 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:25.971+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:26.733 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:26.727+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:26.763 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:26.757+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:26.864 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:26.859+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:26.937 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:26.931+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:26.953 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:26.948+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:27.714 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:27.709+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:27.807 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:27.800+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:27.899 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:27.893+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:27.935 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:27.930+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:27.946 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:27.942+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:27.951 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:27.945+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:27.972 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:27.963+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:28.673 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:28.667+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:28.789 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:28.782+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:28.896 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:28.891+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:28.920 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:28.914+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:28.957 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:28.951+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:28.958 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:28.954+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:28.978 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:28.972+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:29.689 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:29.684+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:29.833 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:29.827+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:29.873 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:29.868+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:29.894 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:29.885+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:29.967 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:29.962+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:29.995 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:29.990+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:30.016 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:30.010+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:30.660 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:30.653+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:30.805 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:30.800+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:30.905 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:30.901+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:30.912 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:30.905+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:30.972 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:30.967+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:30.990 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:30.984+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:31.017 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:31.009+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:31.668 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:31.662+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:31.806 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:31.800+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:31.887 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:31.882+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:31.942 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:31.937+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:31.944 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:31.939+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:32.012 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:32.005+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:32.055 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:32.050+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:32.685 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:32.679+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:32.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:32.848+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:32.859 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:32.848+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:32.926 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:32.921+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:32.951 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:32.947+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:33.031 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:33.025+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:33.067 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:33.062+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:33.677 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:33.672+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:33.884 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:33.878+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:33.890 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:33.885+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:33.950 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:33.944+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:34.004 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:33.998+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:34.028 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:34.022+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:34.095 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:34.089+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:34.654 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:34.648+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:34.862 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:34.857+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:34.883 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:34.876+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:34.971 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:34.965+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:34.991 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:34.985+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:34.992 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:34.985+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:35.072 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:35.066+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:35.704 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:35.698+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:35.877 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:35.872+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:35.936 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:35.931+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:35.948 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:35.942+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:35.980 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:35.974+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:35.995 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:35.990+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:36.103 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:36.099+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:36.676 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:36.670+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:36.914 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:36.909+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:36.918 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:36.912+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:36.931 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:36.925+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:36.968 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:36.963+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:36.989 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:36.984+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:37.080 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:37.074+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:37.697 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:37.690+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:37.902 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:37.897+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:37.912 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:37.906+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:37.934 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:37.928+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:37.980 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:37.974+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:37.993 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:37.988+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:38.045 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:38.040+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:38.725 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:38.719+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:38.906 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:38.899+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:38.948 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:38.944+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:38.965 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:38.959+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:38.974 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:38.969+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:39.005 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:39.000+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:39.041 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:39.036+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:39.779 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:39.773+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:39.910 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:39.905+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:39.912 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:39.906+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:39.950 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:39.946+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:39.954 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:39.948+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:40.008 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:40.002+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:40.028 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:40.024+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:40.762 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:40.757+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:40.892 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:40.886+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:40.924 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:40.919+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:40.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:40.926+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:40.973 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:40.969+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:41.030 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:41.024+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:41.042 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:41.038+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:41.791 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:41.785+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:41.885 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:41.879+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:41.922 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:41.915+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:41.934 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:41.927+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:41.966 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:41.959+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:41.990 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:41.984+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:42.048 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:42.042+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:42.823 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:42.817+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:42.848 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:42.842+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:42.921 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:42.915+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:42.958 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:42.952+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:43.001 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:42.993+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:43.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:43.001+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:43.018 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:43.010+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:43.828 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:43.820+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:43.848 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:43.843+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:43.944 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:43.937+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:43.972 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:43.966+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:43.978 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:43.972+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:43.999 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:43.993+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:44.050 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:44.044+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:44.828 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:44.822+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:44.836 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:44.831+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:44.943 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:44.936+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:44.997 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:44.991+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:45.001 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:44.995+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:45.004 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:44.998+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:45.012 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:45.004+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:45.800 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:45.794+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:45.820 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:45.815+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:45.914 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:45.908+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:45.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:45.970+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:45.981 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:45.975+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:46.001 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:45.994+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:46.028 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:46.022+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:46.804 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:46.795+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:46.812 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:46.805+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:46.942 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:46.936+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:46.955 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:46.949+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:47.009 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:47.003+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:47.046 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:47.041+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:47.067 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:47.061+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:47.769 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:47.763+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:47.835 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:47.830+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:47.925 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:47.914+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:47.965 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:47.958+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:48.029 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:48.024+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:48.037 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:48.032+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:48.039 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:48.033+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:48.813 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:48.808+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:48.865 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:48.860+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:48.936 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:48.929+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:48.971 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:48.966+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:48.986 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:48.980+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:49.066 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:49.059+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:49.076 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:49.070+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:49.794 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:49.787+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:49.887 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:49.875+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:49.942 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:49.936+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:49.977 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:49.971+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:50.034 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:50.027+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:50.073 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:50.068+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:50.091 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:50.084+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:50.796 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:50.790+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:50.883 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:50.878+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:50.903 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:50.897+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:50.974 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:50.968+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:51.015 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:51.009+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:51.041 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:51.033+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:51.144 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:51.138+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:51.835 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:51.828+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:51.894 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:51.889+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:51.931 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:51.926+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:52.022 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:52.016+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:52.042 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:52.036+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:52.048 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:52.043+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:52.185 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:52.178+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:52.803 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:52.796+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:52.938 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:52.932+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:52.945 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:52.939+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:53.001 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:52.995+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:53.047 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:53.040+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:53.058 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:53.052+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:53.197 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:53.190+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:53.790 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:53.784+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:53.899 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:53.893+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:53.973 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:53.966+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:54.003 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:53.996+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:54.008 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:54.001+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:54.018 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:54.012+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:54.156 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:54.149+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:54.815 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:54.809+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:54.932 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:54.926+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:54.937 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:54.931+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:54.974 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:54.968+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:55.044 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:55.038+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:55.048 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:55.042+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:55.183 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:55.177+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:55.866 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:55.860+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:55.932 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:55.926+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:55.960 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:55.954+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:55.994 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:55.988+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:56.031 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:56.025+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:56.056 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:56.049+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:56.148 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:56.141+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:56.835 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:56.831+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:56.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:56.938+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:56.982 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:56.977+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:57.000 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:56.985+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:57.059 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:57.056+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:57.092 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:57.086+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:57.148 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:57.143+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:57.823 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:57.818+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:57.946 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:57.941+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:57.980 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:57.974+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:57.996 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:57.990+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:58.025 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:58.021+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:58.110 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:58.105+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:58.188 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:58.182+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:58.804 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:58.799+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:58.919 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:58.914+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:58.990 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:58.985+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:15:59.046 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:15:59.040+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:15:59.053 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:15:59.048+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:15:59.132 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:15:59.127+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:15:59.146 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:15:59.139+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:15:59.786 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:15:59.781+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:15:59.916 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:15:59.911+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:15:59.963 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:15:59.958+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:16:00.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:16:00.071+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:16:00.105 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:16:00.099+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:16:00.116 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:16:00.109+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:16:00.183 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:16:00.177+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:16:00.489 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:16:00.483+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e223 v223) 2019-12-09T18:16:00.804 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:16:00.797+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:16:00.947 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:16:00.942+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:16:01.012 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:16:01.007+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:16:01.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:16:01.098+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:16:01.120 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:16:01.114+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:16:01.123 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:16:01.119+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:16:01.237 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:16:01.229+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:16:01.813 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:16:01.809+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:16:01.918 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:16:01.914+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:16:02.061 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:16:02.057+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:16:02.064 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:16:02.057+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:16:02.135 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:16:02.128+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:16:02.162 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:16:02.157+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:16:02.284 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:16:02.278+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:16:02.842 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:16:02.838+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:16:02.971 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:16:02.966+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:16:03.054 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:16:03.048+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:16:03.084 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:16:03.079+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:16:03.111 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:16:03.103+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:16:03.136 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:16:03.130+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:16:03.320 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:16:03.315+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:16:03.859 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:16:03.854+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:16:04.016 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:16:04.011+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:16:04.017 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:16:04.012+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:16:04.054 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:16:04.049+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:16:04.136 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:16:04.129+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:16:04.139 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:16:04.135+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:16:04.314 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:16:04.308+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:16:04.838 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:16:04.834+0000 1694f700 -1 osd.6 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:39.940377+0000 front 2019-12-09T18:14:39.963240+0000 (oldest deadline 2019-12-09T18:15:24.538107+0000) 2019-12-09T18:16:05.010 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:16:05.005+0000 1694f700 -1 osd.7 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:44.372441+0000 front 2019-12-09T18:14:44.617098+0000 (oldest deadline 2019-12-09T18:15:27.563022+0000) 2019-12-09T18:16:05.014 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:16:05.010+0000 1694f700 -1 osd.5 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.590557+0000 front 2019-12-09T18:14:42.610866+0000 (oldest deadline 2019-12-09T18:15:25.666387+0000) 2019-12-09T18:16:05.042 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:16:05.035+0000 1694f700 -1 osd.2 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.056766+0000 front 2019-12-09T18:14:41.448450+0000 (oldest deadline 2019-12-09T18:15:27.311944+0000) 2019-12-09T18:16:05.145 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:16:05.140+0000 1694f700 -1 osd.4 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:40.051752+0000 front 2019-12-09T18:14:40.048555+0000 (oldest deadline 2019-12-09T18:15:24.719126+0000) 2019-12-09T18:16:05.154 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:16:05.148+0000 1694f700 -1 osd.1 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:42.057188+0000 front 2019-12-09T18:14:41.550953+0000 (oldest deadline 2019-12-09T18:15:26.238152+0000) 2019-12-09T18:16:05.296 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:16:05.290+0000 1694f700 -1 osd.0 223 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:14:41.292090+0000 front 2019-12-09T18:14:41.543371+0000 (oldest deadline 2019-12-09T18:15:26.592286+0000) 2019-12-09T18:16:05.550 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:16:05.543+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e223 v223) 2019-12-09T18:16:24.532 INFO:teuthology.orchestra.run.smithi059.stderr:testUnorderedList (test.functional.tests.TestContainer) ... ok 2019-12-09T18:16:52.095 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:16:51.916+0000 34274700 -1 osd.3 229 build_incremental_map_msg missing incremental map 228 2019-12-09T18:16:52.226 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:16:51.944+0000 124b0700 -1 osd.3 229 build_incremental_map_msg missing incremental map 228 2019-12-09T18:16:56.232 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:16:56.229+0000 36a79700 -1 osd.3 229 build_incremental_map_msg missing incremental map 228 2019-12-09T18:16:56.779 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:16:56.745+0000 3727a700 -1 osd.3 229 build_incremental_map_msg missing incremental map 228 2019-12-09T18:17:02.741 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:02.735+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:02.795 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:02.790+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:03.246 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:03.240+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:03.736 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:03.730+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:03.789 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:03.784+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:04.260 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:04.255+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:04.705 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:04.700+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:04.737 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:04.732+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:04.770 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:04.765+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:05.244 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:05.237+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:05.431 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:05.425+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:05.690 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:05.684+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:05.716 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:05.711+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:05.811 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:05.806+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:05.863 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:05.858+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:06.257 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:06.251+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:06.426 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:06.421+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:06.713 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:06.699+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:06.735 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:06.728+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:06.779 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:06.774+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:06.908 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:06.901+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:07.296 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:07.289+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:07.467 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:07.461+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:07.714 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:07.708+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:07.737 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:07.731+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:07.828 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:07.821+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:07.917 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:07.911+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:08.263 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:08.258+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:08.503 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:08.497+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:08.702 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:08.696+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:08.752 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:08.745+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:08.841 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:08.835+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:08.933 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:08.927+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:09.219 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:09.213+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:09.489 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:09.482+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:09.728 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:09.722+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:09.799 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:09.793+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:09.870 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:09.865+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:09.929 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:09.924+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:10.213 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:10.207+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:10.491 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:10.486+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:10.747 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:10.741+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:10.792 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:10.787+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:10.915 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:10.910+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:10.927 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:10.922+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:11.188 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:11.183+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:11.531 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:11.526+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:11.771 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:11.765+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:11.808 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:11.803+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:11.880 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:11.873+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:11.908 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:11.902+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:12.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:12.199+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:12.554 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:12.549+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:12.815 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:12.809+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:12.841 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:12.835+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:12.852 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:12.847+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:12.930 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:12.924+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:13.171 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:13.165+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:13.554 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:13.549+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:13.850 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:13.844+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:13.895 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:13.889+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:13.896 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:13.891+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:13.917 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:13.911+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:14.201 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:14.194+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:14.550 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:14.544+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:14.857 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:14.851+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:14.910 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:14.905+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:14.944 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:14.938+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:14.951 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:14.947+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:15.253 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:15.247+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:15.553 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:15.547+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:15.889 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:15.882+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:15.894 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:15.889+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:15.944 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:15.938+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:15.968 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:15.963+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:16.253 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:16.247+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:16.517 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:16.512+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:16.898 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:16.892+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:16.921 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:16.915+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:16.958 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:16.953+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:17.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:17.000+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:17.252 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:17.246+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:17.536 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:17.531+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:17.926 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:17.918+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:17.941 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:17.935+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:18.006 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:18.001+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:18.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:18.048+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:18.297 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:18.291+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:18.579 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:18.574+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:18.786 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:18.781+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:18.914 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:18.909+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:18.970 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:18.965+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:18.988 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:18.982+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:19.026 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:19.020+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:19.321 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:19.315+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:19.621 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:19.616+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:19.769 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:19.763+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:19.890 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:19.883+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:20.016 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:20.010+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:20.033 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:20.029+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:20.078 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:20.071+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:20.293 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:20.281+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:20.591 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:20.585+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:20.822 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:20.816+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:20.882 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:20.877+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:21.049 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:21.041+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:21.061 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:21.056+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:21.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:21.062+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:21.326 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:21.321+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:21.547 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:21.542+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:21.832 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:21.827+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:21.851 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:21.846+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:22.016 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:22.010+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:22.100 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:22.094+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:22.112 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:22.106+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:22.346 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:22.341+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:22.514 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:22.509+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:22.802 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:22.796+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:22.895 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:22.890+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:23.022 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:23.017+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:23.103 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:23.098+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:23.107 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:23.101+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:23.393 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:23.388+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:23.534 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:23.529+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:23.825 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:23.820+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:23.877 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:23.870+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:24.012 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:24.006+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:24.137 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:24.133+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:24.144 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:24.139+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:24.372 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:24.367+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:24.561 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:24.554+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:24.837 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:24.831+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:24.920 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:24.914+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:25.028 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:25.022+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:25.153 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:25.148+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:25.186 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:25.180+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:25.372 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:25.366+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:25.546 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:25.541+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:25.875 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:25.867+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:25.876 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:25.870+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:25.954 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:25.949+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:26.044 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:26.037+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:26.140 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:26.135+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:26.198 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:26.193+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:26.326 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:26.321+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:26.513 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:26.507+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:26.832 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:26.826+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:26.835 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:26.830+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:26.948 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:26.942+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:27.071 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:27.065+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:27.103 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:27.099+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:27.227 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:27.222+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:27.334 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:27.328+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:27.558 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:27.553+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:27.862 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:27.855+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:27.864 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:27.857+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:27.953 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:27.947+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:28.071 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:28.067+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:28.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:28.086+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:28.229 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:28.224+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:28.309 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:28.303+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:28.520 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:28.516+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:28.894 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:28.889+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:28.897 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:28.892+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:28.909 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:28.904+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:29.029 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:29.025+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:29.090 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:29.084+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:29.188 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:29.184+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:29.353 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:29.347+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:29.484 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:29.479+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:29.857 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:29.851+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:29.860 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:29.855+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:29.929 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:29.922+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:30.068 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:30.064+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:30.116 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:30.109+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:30.239 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:30.234+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:30.395 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:30.390+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:30.494 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:30.490+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:30.829 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:30.822+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:30.830 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:30.824+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:30.954 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:30.947+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:31.026 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:31.022+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:31.138 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:31.132+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:31.237 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:31.231+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:31.405 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:31.399+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:31.537 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:31.533+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:31.848 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:31.840+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:31.848 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:31.843+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:31.942 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:31.936+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:32.070 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:32.066+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:32.140 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:32.132+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:32.272 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:32.268+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:32.366 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:32.361+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:32.575 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:32.571+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:32.869 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:32.863+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:32.870 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:32.864+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:32.972 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:32.963+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:33.057 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:33.053+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:33.177 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:33.170+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:33.309 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:33.305+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:33.359 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:33.352+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:33.591 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:33.587+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:33.832 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:33.827+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:33.835 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:33.830+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:33.961 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:33.956+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:34.097 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:34.092+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:34.211 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:34.204+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:34.347 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:34.337+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:34.360 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:34.354+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:34.589 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:34.585+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:34.830 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:34.823+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:34.831 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:34.825+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:34.977 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:34.970+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:35.084 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:35.079+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:35.249 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:35.243+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:35.359 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:35.349+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:35.373 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:35.370+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:35.569 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:35.564+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:35.860 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:35.855+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:35.863 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:35.858+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:35.883 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:17:35.874+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e229 v229) 2019-12-09T18:17:35.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:35.936+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:36.075 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:36.071+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:36.233 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:36.227+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:36.339 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:36.335+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:36.402 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:36.397+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:36.546 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:36.541+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:36.833 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:36.827+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:36.859 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:36.831+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:36.907 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:36.901+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:37.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:37.046+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:37.196 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:37.190+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:37.337 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:37.332+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:37.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:37.394+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:37.565 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:37.560+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:37.822 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:37.816+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:37.823 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:37.817+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:37.872 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:37.866+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:38.057 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:38.052+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:38.243 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:38.235+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:38.308 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:38.301+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:38.446 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:38.440+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:38.595 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:38.590+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:38.866 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:38.860+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:38.866 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:38.861+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:38.897 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:38.891+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:39.040 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:39.035+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:39.218 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:39.212+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:39.348 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:39.341+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:39.491 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:39.486+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:39.576 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:39.570+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:39.840 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:39.835+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:39.843 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:39.838+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:39.874 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:39.869+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:40.007 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:40.002+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:40.218 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:40.213+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:40.319 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:40.314+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:40.534 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:40.529+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:40.562 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:40.556+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:40.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:40.845+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:40.854 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:40.847+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:40.856 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:40.851+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:40.896 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:17:40.890+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e229 v229) 2019-12-09T18:17:40.974 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:40.969+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:41.250 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:41.244+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:41.311 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:41.307+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:41.532 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:41.527+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:41.552 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:41.547+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:41.832 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:41.826+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:41.835 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:41.830+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:41.881 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:41.875+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:41.960 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:41.954+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:42.222 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:42.216+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:42.368 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:42.362+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:42.508 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:42.501+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:42.527 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:42.523+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:42.830 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:42.825+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:42.834 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:42.829+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:42.865 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:42.859+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:42.979 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:42.973+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:43.270 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:43.263+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:43.325 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:43.320+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:43.493 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:43.487+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:43.507 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:43.501+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:43.799 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:43.793+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:43.800 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:43.795+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:43.922 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:43.912+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:44.022 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:44.016+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:44.244 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:44.238+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:44.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:44.341+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:44.486 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:44.481+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:44.548 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:44.542+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:44.822 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:44.815+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:44.823 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:44.817+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:44.970 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:44.964+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:45.013 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:45.007+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:45.257 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:45.251+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:45.396 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:45.390+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:45.449 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:45.442+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:45.518 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:45.511+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:45.800 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:45.793+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:45.800 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:45.795+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:45.918 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:17:45.907+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e229 v229) 2019-12-09T18:17:45.992 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:45.987+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:46.007 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:46.001+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:46.267 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:46.261+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:46.438 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:46.432+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:46.455 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:46.448+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:46.538 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:46.531+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:46.812 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:46.805+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:46.813 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:46.807+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:46.996 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:46.990+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:47.033 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:47.027+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:47.275 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:47.269+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:47.443 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:47.438+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:47.483 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:47.478+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:47.586 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:47.580+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:47.863 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:47.856+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:47.863 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:47.858+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:47.977 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:47.972+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:48.030 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:48.024+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:48.236 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:48.230+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:48.486 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:48.481+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:48.527 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:48.522+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:48.627 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:48.621+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:48.872 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:48.868+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:48.875 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:48.872+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:49.001 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:48.996+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:49.030 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:49.025+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:49.226 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:49.221+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:49.440 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:49.435+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:49.523 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:49.517+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:49.656 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:49.651+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:49.830 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:49.825+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:49.830 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:49.826+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:50.052 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:50.045+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:50.093 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:50.087+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:50.215 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:50.211+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:50.473 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:50.468+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:50.484 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:50.479+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:50.697 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:50.693+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:50.800 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:50.794+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:50.800 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:50.796+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:50.932 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:17:50.928+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e229 v229) 2019-12-09T18:17:51.045 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:51.041+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:51.131 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:51.126+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:51.183 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:51.178+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:51.458 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:51.454+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:51.490 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:51.486+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:51.667 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:51.662+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:51.782 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:51.777+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:51.785 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:51.781+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:52.011 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:52.006+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:52.169 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:52.164+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:52.207 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:52.202+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:52.422 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:52.418+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:52.507 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:52.502+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:52.661 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:52.657+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:52.756 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:52.751+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:52.759 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:52.755+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:52.993 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:52.987+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:53.181 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:53.176+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:53.232 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:53.226+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:53.434 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:53.429+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:53.519 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:53.515+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:53.631 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:53.626+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:53.777 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:53.773+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:53.780 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:53.776+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:53.985 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:53.979+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:54.182 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:54.176+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:54.211 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:54.207+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:54.400 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:54.395+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:54.556 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:54.551+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:54.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:54.657+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:54.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:54.762+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:54.772 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:54.764+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:55.017 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:55.013+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:55.178 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:55.172+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:55.191 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:55.187+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:55.362 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:55.352+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:55.582 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:55.577+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:55.625 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:55.620+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:55.818 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:55.814+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:55.820 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:55.817+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:55.957 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:17:55.951+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e229 v229) 2019-12-09T18:17:56.026 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:56.022+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:56.205 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:56.201+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:56.242 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:56.239+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:56.389 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:56.385+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:56.583 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:56.578+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:56.614 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:56.609+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:56.789 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:56.784+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:56.790 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:56.785+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:57.061 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:57.057+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:57.233 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:57.228+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:57.242 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:57.238+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:57.416 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:57.410+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:57.635 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:57.631+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:57.656 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:57.652+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:57.839 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:57.833+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:57.840 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:57.835+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:58.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:58.109+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:58.229 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:58.224+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:58.270 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:58.265+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:58.382 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:58.377+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:58.690 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:58.684+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:58.709 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:58.704+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:58.794 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:58.789+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:58.795 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:58.790+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:17:59.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:17:59.110+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:17:59.272 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:17:59.267+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:17:59.327 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:17:59.321+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:17:59.435 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:17:59.430+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:17:59.720 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:17:59.714+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:17:59.751 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:17:59.746+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:17:59.780 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:59.774+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:17:59.781 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:17:59.776+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:18:00.157 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:18:00.152+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:18:00.289 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:18:00.282+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.836862+0000 front 2019-12-09T18:16:19.646260+0000 (oldest deadline 2019-12-09T18:17:04.184884+0000) 2019-12-09T18:18:00.296 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:18:00.288+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:21.452993+0000 front 2019-12-09T18:16:22.932497+0000 (oldest deadline 2019-12-09T18:17:05.843320+0000) 2019-12-09T18:18:00.433 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:18:00.428+0000 1694f700 -1 osd.5 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:22.983400+0000 front 2019-12-09T18:16:22.993677+0000 (oldest deadline 2019-12-09T18:17:02.188828+0000) 2019-12-09T18:18:00.736 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:18:00.731+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:18.705210+0000 front 2019-12-09T18:16:21.529873+0000 (oldest deadline 2019-12-09T18:17:03.136860+0000) 2019-12-09T18:18:00.758 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:18:00.753+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:19.051604+0000 front 2019-12-09T18:16:21.437298+0000 (oldest deadline 2019-12-09T18:17:04.930052+0000) 2019-12-09T18:18:00.786 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:18:00.781+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:16:45.846463+0000 (oldest deadline 2019-12-09T18:17:25.846463+0000) 2019-12-09T18:18:00.787 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:18:00.782+0000 1694f700 -1 osd.3 229 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:18:01.043 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:18:01.035+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e229 v229) 2019-12-09T18:18:01.130 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:18:01.125+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:16:17.657808+0000 front 2019-12-09T18:16:17.496516+0000 (oldest deadline 2019-12-09T18:17:02.644795+0000) 2019-12-09T18:18:02.058 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:18:01.906+0000 1694f700 -1 osd.3 230 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e227) 2019-12-09T18:18:56.384 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:35:53.003 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.392 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:35:53.012 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.400 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:35:53.019 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.406 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:35:53.024 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.425 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:35:53.024 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.437 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:35:53.035 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.688 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:35:53.307 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.695 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:35:53.314 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:18:56.824 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:35:53.423 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:19:34.480 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:34.471+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:34.516 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:34.510+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:34.826 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:34.820+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:35.301 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:35.295+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:35.523 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:35.517+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:35.566 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:35.559+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:35.668 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:35.663+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:35.822 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:35.815+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:36.309 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:36.303+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:36.467 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:36.460+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:36.487 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:36.476+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:36.583 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:36.577+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:36.627 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:19:36.621+0000 1694f700 -1 osd.2 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:53.071071+0000 front 2019-12-09T18:18:53.051257+0000 (oldest deadline 2019-12-09T18:19:36.532670+0000) 2019-12-09T18:19:36.652 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:36.646+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:36.813 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:36.807+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:37.318 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:37.311+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:37.488 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:37.482+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:37.507 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:37.501+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:37.567 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:37.562+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:37.696 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:37.690+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:37.841 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:37.836+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:38.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:38.267+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:38.461 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:38.454+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:38.465 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:38.458+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:38.601 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:38.595+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:38.705 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:38.697+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:38.811 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:38.805+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:39.270 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:39.264+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:39.458 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:39.450+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:39.496 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:39.487+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:39.639 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:39.633+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:39.716 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:39.710+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:39.846 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:39.841+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:40.254 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:40.248+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:40.461 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:40.455+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:40.508 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:40.500+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:40.633 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:40.627+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:40.677 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:40.670+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:40.839 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:40.833+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:41.236 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:41.231+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:41.425 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:41.419+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:41.550 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:41.544+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:41.674 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:41.668+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:41.693 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:41.687+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:41.854 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:41.848+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:42.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:42.277+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:42.436 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:42.431+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:42.572 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:42.566+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:42.645 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:42.639+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:42.704 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:42.699+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:42.810 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:42.804+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:43.302 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:43.297+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:43.394 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:43.388+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:43.619 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:43.614+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:43.637 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:43.631+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:43.664 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:43.660+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:43.840 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:43.833+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:44.264 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:44.258+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:44.426 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:44.421+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:44.610 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:44.604+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:44.660 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:44.653+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:44.669 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:44.665+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:44.869 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:44.863+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:45.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:45.278+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:45.454 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:45.449+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:45.630 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:45.625+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:45.657 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:45.653+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:45.725 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:45.720+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:45.887 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:45.880+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:46.326 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:46.322+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:46.480 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:46.474+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:46.636 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:46.629+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:46.661 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:46.655+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:46.715 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:46.710+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:46.855 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:46.850+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:47.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:47.329+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:47.471 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:47.466+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:47.615 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:47.609+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:47.622 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:47.618+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:47.714 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:47.710+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:47.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:47.873+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:48.341 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:48.336+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:48.496 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:48.491+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:48.632 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:48.626+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:48.649 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:48.644+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:48.758 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:48.752+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:48.861 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:48.856+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:49.391 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:49.383+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:49.536 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:49.529+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:49.650 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:49.645+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:49.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:49.666+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:49.758 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:49.753+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:49.849 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:49.843+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:50.429 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:50.423+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:50.511 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:50.506+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:50.662 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:50.657+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:50.714 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:50.707+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:50.743 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:50.738+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:50.875 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:50.869+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:51.480 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:51.473+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:51.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:51.474+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:51.661 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:51.655+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:51.721 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:51.715+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:51.767 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:51.761+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:51.861 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:51.855+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:52.436 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:52.430+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:52.465 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:52.459+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:52.622 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:52.616+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:52.676 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:52.671+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:52.768 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:52.762+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:52.869 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:52.862+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:53.423 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:53.417+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:53.481 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:53.475+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:53.603 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:53.597+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:53.721 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:53.714+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:53.784 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:53.778+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:53.906 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:53.900+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:54.400 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:54.394+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:54.524 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:54.519+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:54.560 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:54.555+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:54.741 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:54.736+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:54.759 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:54.754+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:54.934 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:54.926+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:55.400 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:55.395+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:55.542 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:55.537+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:55.564 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:55.558+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:55.729 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:55.723+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:55.775 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:55.769+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:55.947 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:55.941+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:56.427 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:56.422+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:56.513 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:56.507+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:56.579 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:56.573+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:56.712 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:56.704+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:56.808 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:56.802+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:56.936 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:56.930+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:57.393 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:57.387+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:57.529 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:57.524+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:57.553 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:57.547+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:57.708 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:57.702+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:57.853 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:57.848+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:57.988 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:57.982+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:58.359 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:58.353+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:58.528 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:58.522+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:58.532 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:58.526+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:58.713 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:58.707+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:58.857 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:58.851+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:19:58.988 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:19:58.983+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:19:59.336 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:19:59.330+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:19:59.486 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:19:59.481+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:19:59.574 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:19:59.568+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:19:59.731 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:19:59.725+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:19:59.858 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:19:59.852+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:00.041 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:00.035+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:00.343 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:00.336+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:00.537 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:00.530+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:00.622 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:00.616+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:00.746 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:00.740+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:00.882 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:00.876+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:01.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:01.079+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:01.370 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:01.363+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:01.544 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:01.535+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:01.608 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:01.603+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:01.716 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:01.710+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:01.898 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:01.893+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:02.104 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:02.098+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:02.412 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:02.406+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:02.519 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:02.514+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:02.650 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:02.644+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:02.734 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:02.729+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:02.865 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:02.858+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:03.117 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:03.110+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:03.390 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:03.383+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:03.571 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:03.565+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:03.680 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:03.674+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:03.758 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:03.752+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:03.829 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:03.822+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:04.138 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:04.132+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:04.427 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:04.420+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:04.563 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:04.557+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:04.697 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:04.692+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:04.712 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:04.707+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:04.841 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:04.835+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:05.175 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:05.169+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:05.420 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:05.415+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:05.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:05.542+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:05.719 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:05.712+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:05.722 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:05.717+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:05.806 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:05.801+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:06.155 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:06.148+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:06.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:06.412+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:06.456 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:20:06.434+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e235 v235) 2019-12-09T18:20:06.569 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:06.562+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:06.740 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:06.732+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:06.770 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:06.765+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:06.797 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:06.791+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:07.110 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:07.103+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:07.401 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:07.395+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:07.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:07.542+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:07.711 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:07.705+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:07.757 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:07.751+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:07.825 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:07.820+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:08.092 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:08.085+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:08.427 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:08.422+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:08.558 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:08.552+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:08.670 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:08.664+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:08.801 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:08.795+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:08.823 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:08.817+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:09.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:09.129+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:09.444 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:09.438+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:09.579 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:09.573+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:09.711 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:09.704+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:09.812 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:09.806+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:09.835 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:09.829+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:10.175 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:10.169+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:10.459 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:10.454+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:10.575 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:10.569+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:10.763 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:10.744+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:10.812 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:10.806+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:10.859 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:10.853+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:11.208 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:11.202+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:11.433 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:11.428+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:11.467 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:20:11.463+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e235 v235) 2019-12-09T18:20:11.565 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:11.560+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:11.733 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:11.727+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:11.831 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:11.824+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:11.911 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:11.905+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:12.191 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:12.185+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:12.470 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:12.464+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:12.598 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:12.591+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:12.733 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:12.728+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:12.824 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:12.818+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:12.934 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:12.928+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:13.163 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:13.157+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:13.484 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:13.478+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:13.623 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:13.618+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:13.718 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:13.711+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:13.872 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:13.866+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:13.893 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:13.887+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:14.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:14.129+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:14.488 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:14.482+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:14.611 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:14.605+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:14.673 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:14.667+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:14.830 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:14.823+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:14.937 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:14.932+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:15.170 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:15.164+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:15.544 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:15.538+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:15.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:15.618+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:15.655 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:15.648+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:15.807 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:15.802+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:15.908 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:15.902+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:16.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:16.206+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.975832+0000 front 2019-12-09T18:18:53.007587+0000 (oldest deadline 2019-12-09T18:19:34.647607+0000) 2019-12-09T18:20:16.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:16.543+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.100063+0000 front 2019-12-09T18:18:52.098333+0000 (oldest deadline 2019-12-09T18:19:34.943813+0000) 2019-12-09T18:20:16.615 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:20:16.599+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e235 v235) 2019-12-09T18:20:16.635 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:20:16.629+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:52.622421+0000 front 2019-12-09T18:18:52.634891+0000 (oldest deadline 2019-12-09T18:19:33.623980+0000) 2019-12-09T18:20:16.647 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:16.641+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:51.035565+0000 front 2019-12-09T18:18:51.322406+0000 (oldest deadline 2019-12-09T18:19:35.695044+0000) 2019-12-09T18:20:16.817 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:20:16.812+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:49.680078+0000 front 2019-12-09T18:18:49.678791+0000 (oldest deadline 2019-12-09T18:19:35.548017+0000) 2019-12-09T18:20:16.922 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:20:16.917+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:18:50.330163+0000 front 2019-12-09T18:18:49.859969+0000 (oldest deadline 2019-12-09T18:19:33.926417+0000) 2019-12-09T18:20:46.902 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:46.897+0000 1694f700 -1 osd.0 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:47.858 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:47.854+0000 1694f700 -1 osd.0 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:48.890 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:48.886+0000 1694f700 -1 osd.0 238 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:49.963 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:49.955+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:50.967 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:50.963+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:51.962 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:51.958+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:52.980 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:52.972+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:53.979 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:53.974+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:55.020 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:55.016+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:56.064 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:56.057+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:57.029 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:57.024+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:58.017 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:58.013+0000 1694f700 -1 osd.0 239 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:58.636 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:58.616+0000 36a79700 -1 osd.5 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:58.709 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:58.690+0000 34274700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:58.816 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:58.756+0000 3827c700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:58.909 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:58.777+0000 37a7b700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:58.932 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:58.723+0000 36a79700 -1 osd.5 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:58.938 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:58.789+0000 37a7b700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:58.977 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:58.800+0000 3827c700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:59.010 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:58.781+0000 34274700 -1 osd.5 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:59.029 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:58.843+0000 34a75700 -1 osd.5 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:59.038 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:58.916+0000 34a75700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:59.056 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:20:58.926+0000 3aa81700 -1 osd.5 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:20:59.167 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:20:59.147+0000 1694f700 -1 osd.0 240 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121132 2.7s1 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e236) 2019-12-09T18:20:59.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:20:59.409+0000 3aa81700 -1 osd.4 240 build_incremental_map_msg missing incremental map 240 2019-12-09T18:21:08.701 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:21:08.691+0000 20e25700 -1 RGWWatcher::handle_error cookie 5418615888 err (107) Transport endpoint is not connected 2019-12-09T18:21:12.215 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:12.209+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:12.578 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:12.574+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:13.221 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:13.214+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:13.583 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:13.580+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:14.192 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:14.186+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:14.567 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:14.562+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:14.568 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:14.563+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:14.923 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:14.920+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:15.233 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:15.228+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:15.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:15.543+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:15.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:15.544+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:15.957 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:15.950+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:16.197 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:16.191+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:16.520 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:16.516+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:16.524 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:16.520+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:16.932 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:16.925+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:17.032 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:17.027+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:17.058 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:17.054+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:17.185 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:17.179+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:17.492 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:17.486+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:17.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:17.489+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:17.949 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:17.945+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:18.044 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:18.039+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:18.105 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:18.101+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:18.158 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:18.154+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:18.530 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:18.524+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:18.531 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:18.527+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:18.965 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:18.960+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:19.035 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:19.029+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:19.078 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:19.072+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:19.183 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:19.177+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:19.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:19.483+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:19.503 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:19.494+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:19.962 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:19.958+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:20.042 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:20.036+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:20.130 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:20.124+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:20.213 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:20.207+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:20.505 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:20.500+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:20.506 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:20.502+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:20.972 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:20.967+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:21.049 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:21.043+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:21.103 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:21.099+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:21.201 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:21.196+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:21.541 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:21.530+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:21.541 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:21.535+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:21.991 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:21.986+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:22.042 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:22.036+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:22.131 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:22.126+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:22.229 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:22.223+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:22.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:22.543+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:22.550 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:22.545+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:23.013 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:23.008+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:23.053 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:23.046+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:23.114 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:23.108+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:23.255 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:23.250+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:23.513 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:23.508+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:23.514 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:23.509+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:24.010 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:24.005+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:24.041 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:24.034+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:24.077 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:24.071+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:24.253 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:24.246+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:24.540 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:24.534+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:24.541 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:24.536+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:25.042 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:25.036+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:25.059 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:25.053+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:25.064 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:25.058+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:25.234 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:25.227+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:25.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:25.395+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:25.521 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:25.515+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:25.521 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:25.516+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:26.036 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:26.031+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:26.080 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:26.073+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:26.094 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:26.088+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:26.267 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:26.261+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:26.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:26.444+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:26.485 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:26.480+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:26.487 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:26.483+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:27.027 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:27.020+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:27.064 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:27.058+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:27.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:27.087+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:27.225 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:27.218+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:27.421 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:27.415+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:27.503 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:27.497+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:27.503 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:27.499+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:28.051 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:28.045+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:28.060 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:28.054+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:28.067 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:28.061+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:28.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:28.240+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:28.382 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:28.376+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:28.525 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:28.517+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:28.526 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:28.520+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:29.021 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:29.016+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:29.028 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:29.022+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:29.043 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:29.038+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:29.289 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:29.282+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:29.417 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:29.411+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:29.492 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:29.486+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:29.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:29.488+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:30.015 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:30.010+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:30.030 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:30.025+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:30.032 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:30.027+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:30.275 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:30.269+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:30.458 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:30.451+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:30.485 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:30.476+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:30.485 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:30.480+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:30.741 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:30.731+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:30.980 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:30.975+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:31.003 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:30.997+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:31.015 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:31.009+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:31.238 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:31.231+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:31.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:31.477+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:31.485 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:31.480+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:31.485 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:31.481+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:31.761 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:31.751+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:31.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:31.959+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:31.987 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:31.980+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:32.044 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:32.039+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:32.266 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:32.261+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:32.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:32.484+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:32.490 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:32.485+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:32.492 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:32.485+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:32.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:32.765+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:32.949 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:32.944+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:33.014 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:33.008+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:33.085 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:33.080+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:33.291 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:33.285+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:33.459 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:33.454+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:33.461 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:33.455+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:33.461 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:33.457+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:33.735 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:33.729+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:33.942 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:33.936+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:34.020 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:34.015+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:34.094 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:34.089+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:34.317 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:34.312+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:34.422 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:34.417+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:34.425 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:34.421+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:34.491 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:34.485+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:34.718 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:34.712+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:34.977 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:34.972+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:35.043 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:35.037+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:35.117 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:35.113+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:35.297 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:35.292+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:35.407 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:35.401+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:35.409 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:35.403+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:35.508 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:35.502+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:35.703 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:35.698+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:35.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:35.958+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:36.089 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:36.083+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:36.167 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:36.161+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:36.278 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:36.271+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:36.379 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:36.372+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:36.379 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:36.374+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:36.540 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:36.534+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:36.680 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:36.675+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:37.014 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:37.008+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:37.120 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:37.114+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:37.166 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:37.161+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:37.259 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:37.252+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:37.371 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:37.366+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:37.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:37.370+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:37.593 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:37.586+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:37.665 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:37.655+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:37.665 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:37.658+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:38.040 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:38.036+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:38.159 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:38.153+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:38.199 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:38.194+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:38.242 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:38.236+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:38.360 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:38.355+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:38.364 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:38.360+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:38.578 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:38.573+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:38.656 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:38.649+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:38.656 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:38.651+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:39.052 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:39.047+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:39.137 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:39.131+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:39.171 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:39.164+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:39.220 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:39.213+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:39.381 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:39.367+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:39.381 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:39.369+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:39.620 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:39.614+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:39.638 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:39.631+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:39.638 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:39.633+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:40.083 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:40.077+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:40.139 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:40.133+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:40.206 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:40.200+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:40.258 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:40.252+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:40.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:40.341+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:40.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:40.344+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:40.610 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:40.603+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:40.611 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:40.605+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:40.660 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:40.655+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:41.068 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:41.063+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:41.128 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:41.123+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:41.223 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:41.219+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:41.272 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:41.266+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:41.340 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:41.334+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:41.346 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:41.341+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:41.655 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:41.650+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:41.658 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:41.654+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:41.685 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:41.679+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:42.092 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:42.086+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:42.099 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:42.094+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:42.243 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:42.237+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:42.248 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:42.241+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:42.302 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:42.295+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:42.302 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:42.297+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:42.688 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:42.679+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:42.708 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:42.700+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:42.708 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:42.702+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:43.060 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:43.054+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:43.096 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:43.091+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:43.226 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:43.220+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:43.263 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:43.257+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:43.309 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:43.302+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:43.310 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:43.304+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:43.670 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:43.665+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:43.721 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:43.716+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:43.724 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:43.720+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:44.049 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:44.044+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:44.148 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:44.143+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:44.186 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:44.181+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:44.227 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:44.222+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:44.344 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:44.339+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:44.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:44.343+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:44.707 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:44.702+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:44.761 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:44.757+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:44.763 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:44.759+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:45.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:45.092+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:45.179 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:45.175+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:45.193 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:45.188+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:45.276 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:45.271+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:45.320 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:45.314+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:45.321 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:45.316+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:45.729 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:45.723+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:45.730 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:45.725+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:45.748 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:45.743+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:46.079 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:46.074+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:46.195 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:46.190+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:46.197 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:46.192+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:46.253 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:46.248+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:46.364 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:46.359+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:46.367 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:46.363+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:46.756 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:46.750+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:46.774 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:46.770+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:46.775 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:46.771+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:46.984 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:21:46.977+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e241 v241) 2019-12-09T18:21:47.125 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:47.121+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:47.182 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:47.177+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:47.203 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:47.198+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:47.225 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:47.221+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:47.405 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:47.398+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:47.406 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:47.400+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:47.805 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:47.800+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:47.818 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:47.813+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:47.820 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:47.815+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:48.150 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:48.144+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:48.173 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:48.167+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:48.185 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:48.179+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:48.195 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:48.190+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:48.443 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:48.437+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:48.448 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:48.442+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:48.764 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:48.758+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:48.835 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:48.828+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:48.837 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:48.830+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:49.138 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:49.133+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:49.158 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:49.152+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:49.238 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:49.232+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:49.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:49.242+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:49.460 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:49.454+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:49.461 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:49.456+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:49.798 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:49.793+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:49.822 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:49.818+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:49.826 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:49.821+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:50.131 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:50.127+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:50.181 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:50.176+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:50.247 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:50.243+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:50.266 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:50.261+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:50.477 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:50.470+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:50.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:50.474+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:50.819 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:50.813+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:50.826 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:50.820+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:50.826 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:50.822+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:51.165 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:51.160+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:51.178 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:51.173+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:51.250 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:51.245+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:51.306 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:51.301+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:51.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:51.488+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:51.519 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:51.492+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:51.794 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:51.789+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:51.854 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:51.849+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:51.855 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:51.851+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:52.005 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:21:51.997+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e241 v241) 2019-12-09T18:21:52.145 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:52.140+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:52.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:52.215+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:52.225 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:52.220+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:52.351 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:52.346+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:52.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:52.490+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:52.498 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:52.493+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:52.758 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:52.752+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:52.855 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:52.851+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:52.859 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:52.855+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:53.174 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:53.168+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:53.183 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:53.179+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:53.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:53.214+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:53.317 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:53.312+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:53.468 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:53.461+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:53.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:53.463+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:53.732 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:53.727+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:53.910 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:53.905+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:53.913 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:53.909+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:54.179 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:54.174+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:54.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:54.194+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:54.253 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:54.248+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:54.342 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:54.337+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:54.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:54.492+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:54.501 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:54.496+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:54.781 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:54.777+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:54.924 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:54.919+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:54.924 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:54.921+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:55.162 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:55.158+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:55.179 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:55.174+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:55.223 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:55.218+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:55.300 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:55.295+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:55.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:55.508+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:55.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:55.510+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:55.771 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:55.766+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:55.895 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:55.891+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:55.896 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:55.892+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:56.181 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:56.174+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:56.183 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:56.178+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:56.205 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:56.200+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:56.313 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:56.309+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:56.552 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:56.546+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:56.561 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:56.547+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:56.732 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:56.728+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:56.868 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:56.864+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:56.869 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:56.865+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:57.022 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:21:57.015+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e241 v241) 2019-12-09T18:21:57.153 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:57.149+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:57.212 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:57.207+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:57.231 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:57.225+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:57.351 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:57.347+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:57.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:57.571+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:57.579 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:57.576+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:57.717 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:57.712+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:57.870 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:57.865+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:57.870 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:57.866+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:58.128 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:58.124+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:58.184 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:58.178+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:58.200 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:58.195+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:58.337 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:58.333+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:58.543 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:58.537+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:58.544 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:58.540+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:58.758 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:58.752+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:58.896 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:58.891+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:58.897 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:58.892+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:21:59.171 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:21:59.166+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:21:59.179 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:21:59.175+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:21:59.209 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:21:59.204+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:21:59.297 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:21:59.293+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:21:59.552 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:59.548+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:21:59.556 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:21:59.553+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:21:59.758 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:21:59.753+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:21:59.872 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:59.867+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:21:59.874 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:21:59.871+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:00.140 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:00.136+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:00.173 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:00.167+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:00.262 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:00.257+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:00.292 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:00.288+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:00.572 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:00.566+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:00.572 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:00.568+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:00.780 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:00.775+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:00.837 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:00.825+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:00.837 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:00.827+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:01.146 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:01.141+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:01.177 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:01.170+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:01.270 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:01.265+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:01.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:01.298+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:01.611 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:01.605+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:01.612 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:01.608+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:01.766 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:01.761+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:01.833 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:01.827+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:01.834 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:01.829+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:02.047 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:22:02.039+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e241 v241) 2019-12-09T18:22:02.185 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:02.181+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:02.207 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:02.204+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:02.324 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:02.320+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:02.345 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:02.340+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:02.661 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:02.657+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:02.665 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:02.661+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:02.726 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:02.721+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:02.874 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:02.869+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:02.875 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:02.870+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:03.177 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:03.172+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:03.178 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:03.174+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:03.326 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:03.318+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:03.374 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:03.369+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:03.666 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:03.661+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:03.667 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:03.662+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:03.761 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:03.756+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:03.853 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:03.848+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:03.853 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:03.849+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:04.145 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:04.140+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:04.217 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:04.213+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:04.361 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:04.356+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:04.365 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:04.360+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:04.708 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:04.703+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:04.709 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:04.705+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:04.753 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:04.749+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:04.847 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:04.841+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:04.848 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:04.843+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:05.166 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:05.161+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:05.221 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:05.217+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:05.328 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:05.324+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:05.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:05.375+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:05.725 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:05.720+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:05.726 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:05.722+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:05.764 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:05.759+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:05.804 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:05.799+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:05.805 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:05.800+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:06.204 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:06.199+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:06.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:06.235+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:06.319 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:06.314+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:06.338 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:06.333+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:06.722 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:06.717+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:06.726 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:06.721+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:06.765 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:06.759+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:06.822 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:06.819+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:06.824 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:06.821+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:07.069 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:22:07.065+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e241 v241) 2019-12-09T18:22:07.162 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:07.158+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:07.276 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:07.270+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:07.293 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:07.288+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:07.338 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:07.333+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:07.723 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:07.717+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:07.725 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:07.719+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:07.778 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:07.774+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:07.789 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:07.786+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:07.791 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:07.788+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:08.174 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:08.169+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:08.259 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:08.254+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:08.287 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:08.281+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:08.295 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:08.289+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:08.712 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:08.706+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:08.713 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:08.708+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:08.749 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:08.743+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:08.750 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:08.745+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:08.815 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:08.810+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:09.143 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:09.138+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:09.254 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:09.249+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:09.283 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:09.278+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:09.294 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:09.286+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:09.676 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:09.669+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:09.677 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:09.672+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:09.767 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:09.758+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:09.767 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:09.762+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:09.785 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:09.780+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:10.160 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:10.156+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:10.271 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:10.266+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:10.315 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:10.310+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:10.332 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:10.327+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:10.656 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:10.652+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:10.658 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:10.654+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:10.766 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:10.760+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:10.813 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:10.805+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:10.813 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:10.806+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:11.135 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:11.129+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:11.258 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:11.254+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:11.339 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:11.334+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:11.345 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:11.340+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:11.668 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:11.663+0000 1694f700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.958448+0000 front 2019-12-09T18:20:28.738419+0000 (oldest deadline 2019-12-09T18:21:13.705299+0000) 2019-12-09T18:22:11.672 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:11.667+0000 1694f700 -1 osd.4 241 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:11.769 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:22:11.764+0000 1694f700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.432801+0000 front 2019-12-09T18:20:28.551644+0000 (oldest deadline 2019-12-09T18:21:25.030873+0000) 2019-12-09T18:22:11.784 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:11.779+0000 1694f700 -1 osd.3 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:20:56.866284+0000 (oldest deadline 2019-12-09T18:21:36.866284+0000) 2019-12-09T18:22:11.785 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:22:11.780+0000 1694f700 -1 osd.3 241 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121132 2.7s0 2.c84a5507 (undecoded) ondisk+retry+write+known_if_redirected e240) 2019-12-09T18:22:12.126 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:22:12.120+0000 1694f700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:28.271822+0000 front 2019-12-09T18:20:28.825935+0000 (oldest deadline 2019-12-09T18:21:12.124287+0000) 2019-12-09T18:22:12.151 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:22:12.141+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 44sec e241 v241) 2019-12-09T18:22:12.243 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:22:12.235+0000 1694f700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:34.211476+0000 front 2019-12-09T18:20:31.213526+0000 (oldest deadline 2019-12-09T18:21:16.105866+0000) 2019-12-09T18:22:12.308 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:22:12.303+0000 1694f700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:31.589484+0000 front 2019-12-09T18:20:31.573244+0000 (oldest deadline 2019-12-09T18:21:14.703197+0000) 2019-12-09T18:22:12.342 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:22:12.338+0000 1694f700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:20:32.214848+0000 front 2019-12-09T18:20:32.314802+0000 (oldest deadline 2019-12-09T18:21:16.151037+0000) 2019-12-09T18:22:13.104 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:12.938+0000 1694f700 -1 osd.4 242 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121324 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:14.212 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:22:14.191+0000 1694f700 -1 osd.4 243 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121332 5.3 5:c609908c:::notify.5:head [watch ping cookie 5418615888 gen 3] snapc 0=[] ondisk+write+known_if_redirected e238) 2019-12-09T18:22:48.844 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:39:45.463 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:48.850 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:39:45.469 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:48.864 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:39:45.483 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:48.868 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:39:45.487 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:48.875 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:39:45.473 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:48.894 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:39:45.493 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:49.112 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:39:45.731 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:49.121 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:39:45.740 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:49.308 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:39:45.907 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.481 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:39:54.100 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.489 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:39:54.108 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.517 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:39:54.136 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.552 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:39:54.171 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.559 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:39:54.178 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.578 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:39:54.177 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.581 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:39:54.180 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.588 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:39:54.207 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:57.619 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:39:54.218 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:23:59.256 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:23:59.247+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:23:59.267 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:23:59.261+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:00.217 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:24:00.209+0000 1694f700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.917554+0000 front 2019-12-09T18:23:16.794787+0000 (oldest deadline 2019-12-09T18:23:59.906109+0000) 2019-12-09T18:24:00.258 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:00.252+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:00.294 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:00.288+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:00.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:00.462+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:00.733 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:00.728+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:00.772 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:00.766+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:01.004 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:00.998+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:01.219 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:24:01.213+0000 1694f700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.917554+0000 front 2019-12-09T18:23:16.794787+0000 (oldest deadline 2019-12-09T18:23:59.906109+0000) 2019-12-09T18:24:01.278 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:01.272+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:01.279 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:01.270+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:01.505 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:01.499+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:01.745 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:01.737+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:01.825 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:01.817+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:02.029 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:02.022+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:02.191 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:24:02.184+0000 1694f700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.917554+0000 front 2019-12-09T18:23:16.794787+0000 (oldest deadline 2019-12-09T18:23:59.906109+0000) 2019-12-09T18:24:02.307 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:02.301+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:02.328 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:02.321+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:02.483 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:02.477+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:02.698 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:02.693+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:02.820 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:02.814+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:02.945 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:40:59.564 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:02.970 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:40:59.589 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:02.999 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:40:59.596 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:02.999 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:02.988+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:03.051 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:40:59.650 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:03.069 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:40:59.685 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:03.110 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:40:59.729 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:03.112 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:40:59.732 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:03.142 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:40:59.740 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:03.186 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:40:59.805 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:03.189 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:24:03.184+0000 1694f700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.917554+0000 front 2019-12-09T18:23:16.794787+0000 (oldest deadline 2019-12-09T18:23:59.906109+0000) 2019-12-09T18:24:03.284 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:03.279+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:03.297 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:03.291+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:03.451 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:03.446+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:03.714 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:03.709+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:03.818 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:03.812+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:04.026 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:04.019+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:04.238 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:24:04.232+0000 1694f700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.917554+0000 front 2019-12-09T18:23:16.794787+0000 (oldest deadline 2019-12-09T18:23:59.906109+0000) 2019-12-09T18:24:04.279 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:04.273+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:04.299 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:04.294+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:04.495 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:04.489+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:04.742 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:04.737+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:04.844 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:04.836+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:05.016 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:05.006+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:05.255 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:24:05.249+0000 1694f700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.917554+0000 front 2019-12-09T18:23:16.794787+0000 (oldest deadline 2019-12-09T18:23:59.906109+0000) 2019-12-09T18:24:05.267 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:05.261+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:05.320 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:05.314+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:05.490 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:05.484+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:05.764 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:05.758+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:05.855 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:05.848+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:06.028 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:06.021+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:06.232 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:06.227+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:06.294 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:06.288+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:06.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:06.455+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:06.731 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:06.725+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:06.876 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:06.870+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:06.994 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:06.988+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:07.219 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:07.212+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:07.340 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:07.330+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:07.448 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:07.442+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:07.783 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:07.777+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:07.913 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:07.908+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:07.961 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:07.955+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:08.194 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:08.188+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:08.299 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:08.293+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:08.421 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:08.414+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:08.764 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:08.757+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:08.953 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:08.946+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:08.958 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:08.952+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:09.149 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:09.144+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:09.265 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:09.251+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:09.446 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:09.440+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:09.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:09.730+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:09.925 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:09.918+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:09.999 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:09.991+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:10.169 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:10.161+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:10.215 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:10.209+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:10.444 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:10.439+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:10.759 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:10.752+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:10.945 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:10.939+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:11.040 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:11.034+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:11.179 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:11.173+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:11.182 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:11.176+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:11.491 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:11.486+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:11.773 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:11.763+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:11.943 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:11.933+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:12.031 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:12.026+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:12.211 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:12.206+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:12.222 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:12.217+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:12.539 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:12.534+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:12.744 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:12.739+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:12.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:12.970+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:13.034 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:13.028+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:13.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:13.237+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:13.246 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:13.230+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:13.523 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:13.518+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:13.763 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:13.757+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:13.934 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:13.928+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:14.034 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:14.028+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:14.211 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:14.204+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:14.234 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:14.229+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:14.486 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:14.481+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:14.807 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:14.802+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:14.916 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:14.910+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:15.087 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:15.080+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:15.217 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:15.210+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:15.240 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:15.233+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:15.447 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:15.441+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:15.784 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:15.780+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:15.872 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:15.866+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:16.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:16.127+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:16.215 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:16.205+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:16.222 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:16.214+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:16.412 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:16.406+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:16.793 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:16.788+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:16.840 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:16.834+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:17.144 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:17.138+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:17.187 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:17.181+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:17.264 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:17.259+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:17.428 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:17.421+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:17.827 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:17.823+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:17.845 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:17.839+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:18.148 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:18.142+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:18.189 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:18.181+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:18.299 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:18.295+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:18.474 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:18.469+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:18.813 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:18.809+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:18.841 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:18.835+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:19.156 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:19.151+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:19.236 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:19.231+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:19.268 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:19.263+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:19.436 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:19.430+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:19.816 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:19.811+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:19.857 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:19.850+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:20.198 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:20.192+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:20.268 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:20.263+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:20.291 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:20.286+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:20.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:20.414+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:20.869 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:20.864+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:20.889 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:20.882+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:21.167 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:21.161+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:21.264 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:21.258+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:21.338 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:21.333+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:21.458 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:21.453+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:21.861 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:21.855+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:21.903 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:21.896+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:22.209 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:22.204+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:22.231 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:22.225+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:22.310 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:22.304+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:22.507 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:22.501+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:22.909 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:22.904+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:22.943 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:22.937+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:23.218 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:23.212+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:23.229 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:23.223+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:23.284 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:23.278+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:23.467 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:23.461+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:23.871 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:23.867+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:23.991 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:23.979+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:24.209 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:24.204+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:24.216 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:24.211+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:24.253 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:24.246+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:24.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:24.509+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:24.906 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:24.901+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:25.004 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:24.998+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:25.224 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:25.218+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:25.226 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:25.220+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:25.230 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:25.225+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:25.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:25.472+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:25.871 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:25.865+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:26.036 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:26.030+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:26.217 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:26.205+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:26.260 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:26.255+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:26.270 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:26.262+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:26.488 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:26.482+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:26.846 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:26.841+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:27.011 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:27.005+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:27.230 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:27.224+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:27.235 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:27.231+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:27.307 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:27.302+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:27.511 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:27.505+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:27.833 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:27.828+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:28.044 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:28.037+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:28.208 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:28.203+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:28.266 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:28.259+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:28.285 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:28.279+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:28.552 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:28.546+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:28.881 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:28.875+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:29.066 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:29.060+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:29.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:29.246+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:29.251 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:29.244+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:29.287 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:29.282+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:29.533 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:29.527+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:29.939 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:29.931+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:30.026 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:30.020+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:30.236 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:30.230+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:30.248 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:30.243+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:30.303 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:30.297+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:30.555 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:30.549+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:30.957 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:30.952+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:31.046 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:31.040+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:31.255 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:31.250+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:31.270 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:31.266+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:31.314 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:31.308+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:31.517 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:31.511+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:32.006 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:32.001+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:32.100 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:32.094+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:32.267 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:32.260+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:32.282 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:32.275+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:32.345 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:32.332+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:32.524 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:32.519+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:32.942 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:24:32.938+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e247 v247) 2019-12-09T18:24:33.027 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:33.022+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:33.109 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:33.102+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:33.286 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:33.280+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:33.292 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:33.286+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:33.350 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:33.345+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:33.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:33.543+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:33.993 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:33.986+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:34.117 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:34.112+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:34.302 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:34.295+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:34.335 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:34.330+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:34.369 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:34.363+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:34.545 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:34.539+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:34.988 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:34.984+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:35.092 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:35.087+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:35.258 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:35.253+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:35.347 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:35.341+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:35.356 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:35.350+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:35.510 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:35.504+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:35.995 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:35.990+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:36.103 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:36.098+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:36.261 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:36.256+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:36.338 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:36.333+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:36.375 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:36.370+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:36.530 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:36.525+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:36.958 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:36.953+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:37.154 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:37.148+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:37.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:37.242+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:37.312 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:37.308+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:37.359 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:37.353+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:37.510 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:37.504+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:37.960 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:37.954+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:37.965 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:24:37.961+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e247 v247) 2019-12-09T18:24:38.109 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:38.103+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:38.292 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:38.286+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:38.309 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:38.304+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:38.404 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:38.398+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:38.495 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:38.488+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:38.998 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:38.988+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:39.142 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:39.136+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:39.309 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:39.304+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:39.340 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:39.335+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:39.414 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:39.408+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:39.507 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:39.501+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:39.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:39.970+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:40.195 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:40.189+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:40.349 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:40.344+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:40.356 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:40.351+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:40.424 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:40.418+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:40.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:40.473+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:40.983 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:40.967+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:41.211 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:41.203+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:41.316 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:41.310+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:41.399 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:41.395+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:41.451 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:41.445+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:41.518 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:41.513+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:41.949 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:41.943+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:42.222 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:42.217+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:24:42.333 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:24:42.327+0000 1694f700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:15.502683+0000 front 2019-12-09T18:23:18.937731+0000 (oldest deadline 2019-12-09T18:23:58.521992+0000) 2019-12-09T18:24:42.409 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:24:42.403+0000 1694f700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:18.151859+0000 front 2019-12-09T18:23:17.494813+0000 (oldest deadline 2019-12-09T18:24:00.684034+0000) 2019-12-09T18:24:42.457 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:24:42.451+0000 1694f700 -1 osd.6 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:19.416620+0000 front 2019-12-09T18:23:18.845123+0000 (oldest deadline 2019-12-09T18:23:58.467464+0000) 2019-12-09T18:24:42.491 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:24:42.485+0000 1694f700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.783455+0000 front 2019-12-09T18:23:16.792021+0000 (oldest deadline 2019-12-09T18:23:59.651163+0000) 2019-12-09T18:24:42.911 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:24:42.906+0000 1694f700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.797881+0000 front 2019-12-09T18:23:16.797256+0000 (oldest deadline 2019-12-09T18:24:00.819835+0000) 2019-12-09T18:24:43.043 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:24:43.040+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e247 v247) 2019-12-09T18:24:43.182 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:24:43.177+0000 1694f700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:23:16.790963+0000 front 2019-12-09T18:23:16.670149+0000 (oldest deadline 2019-12-09T18:24:00.643206+0000) 2019-12-09T18:25:22.709 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.685+0000 3827c700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:22.731 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.724+0000 37a7b700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:22.761 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.729+0000 37a7b700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:22.790 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.755+0000 34a75700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:22.891 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.811+0000 124b0700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:23.114 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.876+0000 36a79700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:23.247 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:25:22.984+0000 3b282700 -1 osd.4 251 build_incremental_map_msg missing incremental map 251 2019-12-09T18:25:25.469 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:25.458+0000 33a73700 -1 osd.3 252 build_incremental_map_msg missing incremental map 252 2019-12-09T18:25:26.091 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:26.081+0000 124b0700 -1 osd.3 252 build_incremental_map_msg missing incremental map 252 2019-12-09T18:25:27.637 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:27.623+0000 36a79700 -1 osd.3 252 build_incremental_map_msg missing incremental map 252 2019-12-09T18:25:29.788 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:29.780+0000 3727a700 -1 osd.3 252 build_incremental_map_msg missing incremental map 252 2019-12-09T18:25:55.061 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:55.057+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:25:56.068 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:56.064+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:25:57.067 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:57.063+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:25:58.043 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:58.039+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:25:59.016 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:25:59.011+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:00.053 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:00.049+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:01.089 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:01.084+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:02.055 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:02.051+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:03.062 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:03.057+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:04.092 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:04.088+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:05.145 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:05.140+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:06.141 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:06.137+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:07.182 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:07.178+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:08.231 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:08.225+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:09.189 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:09.184+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:10.230 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:10.225+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:11.259 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:11.254+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:12.305 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:12.300+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:13.293 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:13.289+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:13.345 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:13.340+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:14.345 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:14.341+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:14.399 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:14.393+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:15.305 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:15.301+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:15.446 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:15.440+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:16.337 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:16.332+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:16.480 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:16.475+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:17.311 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:17.307+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:17.451 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:17.445+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:18.304 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:18.299+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:18.466 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:18.461+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:18.493 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:26:18.488+0000 20e25700 -1 RGWWatcher::handle_error cookie 1050419328 err (107) Transport endpoint is not connected 2019-12-09T18:26:19.337 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:19.333+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:19.466 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:19.461+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:20.378 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:20.373+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:20.432 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:20.425+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:21.351 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:21.346+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:21.394 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:21.389+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:22.368 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:22.363+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:22.395 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:22.389+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:23.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:23.344+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:23.425 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:23.419+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:24.150 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:24.145+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:24.366 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:24.359+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:24.366 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:24.361+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:24.450 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:24.444+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:25.153 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:25.144+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:25.329 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:25.323+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:25.412 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:25.407+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:25.413 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:25.408+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:25.442 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:25.437+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:25.859 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:25.854+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:26.145 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:26.138+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:26.365 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:26.359+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:26.413 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:26.404+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:26.429 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:26.423+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:26.430 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:26.425+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:26.543 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:26.536+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:26.823 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:26.818+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:27.171 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:27.165+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:27.389 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:27.386+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:27.442 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:27.437+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:27.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:27.477+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:27.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:27.478+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:27.519 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:27.513+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:27.814 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:27.809+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:28.157 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:28.151+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:28.417 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:28.413+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:28.453 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:28.447+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:28.469 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:28.458+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:28.469 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:28.460+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:28.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:28.482+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:28.490 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:28.484+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:28.490 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:28.480+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:28.828 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:28.825+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:29.187 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:29.180+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:29.445 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:29.441+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:29.483 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:29.477+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:29.498 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:29.491+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:29.499 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:29.494+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:29.500 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:29.494+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:29.501 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:29.496+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:29.504 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:29.499+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:29.869 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:29.865+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:30.170 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:30.165+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:30.476 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:30.469+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:30.477 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:30.470+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:30.487 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:30.482+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:30.506 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:30.496+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:30.527 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:30.524+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:30.528 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:30.525+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:30.551 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:30.545+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:30.838 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:30.834+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:31.212 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:31.206+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:31.528 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:31.518+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:31.529 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:31.513+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:31.529 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:31.521+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:31.529 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:31.515+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:31.536 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:31.531+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:31.559 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:31.556+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:31.561 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:31.558+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:31.836 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:31.832+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:32.209 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:32.203+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:32.516 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:32.508+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:32.517 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:32.510+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:32.527 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:32.519+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:32.533 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:32.524+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:32.533 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:32.526+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:32.535 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:32.530+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:32.572 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:32.568+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:32.853 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:32.848+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:33.193 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:33.185+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:33.504 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:33.497+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:33.505 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:33.498+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:33.506 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:33.500+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:33.525 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:33.519+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:33.553 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:33.548+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:33.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:33.571+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:33.579 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:33.575+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:33.822 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:33.818+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:34.205 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:34.199+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:34.460 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:34.454+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:34.478 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:34.473+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:34.515 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:34.511+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:34.554 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:34.548+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:34.555 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:34.549+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:34.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:34.594+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:34.601 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:34.597+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:34.782 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:34.777+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:35.238 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:35.232+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:35.497 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:35.491+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:35.508 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:35.503+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:35.521 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:35.514+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:35.529 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:35.522+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:35.530 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:35.525+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:35.562 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:35.556+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:35.563 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:35.559+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:35.814 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:35.810+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:36.289 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:36.282+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:36.452 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:36.446+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:36.491 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:36.485+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:36.508 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:36.502+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:36.522 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:36.517+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:36.525 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:36.521+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:36.582 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:36.575+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:36.583 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:36.577+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:36.863 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:36.858+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:37.333 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:37.325+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:37.440 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:37.433+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:37.474 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:37.468+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:37.500 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:37.494+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:37.553 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:37.548+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:37.555 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:37.552+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:37.562 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:37.555+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:37.563 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:37.557+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:37.864 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:37.848+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:38.346 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:38.340+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:38.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:38.394+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:38.450 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:38.444+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:38.488 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:38.481+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:38.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:38.510+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:38.524 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:38.514+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:38.560 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:38.554+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:38.562 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:38.556+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:38.851 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:38.846+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:39.371 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:39.364+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:39.395 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:39.385+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:39.484 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:39.477+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:39.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:39.487+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:39.500 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:39.489+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:39.503 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:39.498+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:39.549 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:39.542+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:39.568 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:39.544+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:39.824 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:39.819+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:40.351 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:40.345+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:40.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:40.408+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:40.452 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:40.447+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:40.456 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:40.451+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:40.460 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:40.454+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:40.463 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:40.457+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:40.508 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:40.500+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:40.509 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:40.502+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:40.849 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:40.843+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:41.344 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:41.335+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:41.445 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:41.438+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:41.452 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:41.446+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:41.453 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:41.448+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:41.470 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:41.460+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:41.505 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:41.500+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:41.561 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:41.548+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:41.561 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:41.550+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:41.873 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:41.869+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:42.307 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:42.300+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:42.420 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:42.413+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:42.441 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:42.435+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:42.473 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:42.467+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:42.473 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:42.468+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:42.509 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:42.504+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:42.555 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:42.548+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:42.556 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:42.550+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:42.907 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:42.902+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:43.325 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:43.319+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:43.388 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:43.382+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:43.468 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:43.463+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:43.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:43.465+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:43.477 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:43.471+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:43.481 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:43.476+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:43.579 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:43.574+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:43.580 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:43.575+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:43.915 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:43.910+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:44.345 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:44.339+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:44.387 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:44.382+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:44.457 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:44.453+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:44.464 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:44.460+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:44.465 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:44.459+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:44.492 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:44.487+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:44.593 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:44.586+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:44.593 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:44.588+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:44.880 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:44.874+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:45.362 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:45.356+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:45.439 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:45.433+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:45.487 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:45.481+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:45.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:45.483+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:45.490 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:45.485+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:45.491 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:45.485+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:45.608 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:45.601+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:45.623 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:45.603+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:45.929 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:45.921+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:46.386 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:46.380+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:46.408 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:46.401+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:46.489 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:46.483+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:46.516 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:46.511+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:46.539 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:46.533+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:46.542 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:46.535+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:46.612 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:46.606+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:46.614 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:46.607+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:46.904 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:46.900+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:47.421 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:47.415+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:47.429 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:47.423+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:47.527 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:47.520+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:47.537 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:47.524+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:47.592 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:47.584+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:47.592 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:47.586+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:47.593 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:47.588+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:47.598 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:47.592+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:47.909 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:47.904+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:48.382 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:48.377+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:48.405 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:48.396+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:48.535 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:48.529+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:48.539 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:48.533+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:48.595 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:48.590+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:48.598 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:48.593+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:48.638 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:48.627+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:48.638 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:48.631+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:48.877 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:48.872+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:49.357 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:49.347+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:49.374 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:49.360+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:49.561 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:49.554+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:49.562 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:49.556+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:49.585 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:49.578+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:49.586 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:49.580+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:49.622 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:49.616+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:49.623 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:49.618+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:49.897 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:49.891+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:50.311 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:50.305+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:50.413 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:50.407+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:50.549 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:50.543+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:50.576 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:50.569+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:50.577 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:50.571+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:50.613 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:50.607+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:50.629 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:50.622+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:50.630 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:50.625+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:50.902 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:50.897+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:51.319 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:51.313+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:51.421 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:51.413+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:51.572 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:51.566+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:51.593 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:51.588+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:51.613 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:51.607+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:51.614 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:51.608+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:51.649 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:51.642+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:51.650 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:51.644+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:51.928 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:51.922+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:52.345 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:52.338+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:52.395 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:52.388+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:52.552 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:52.546+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:52.583 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:52.577+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:52.627 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:52.617+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:52.628 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:52.621+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:52.637 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:52.630+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:52.638 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:52.634+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:52.898 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:52.893+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:53.367 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:53.361+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:53.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:53.393+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:53.574 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:53.568+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:53.588 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:53.583+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:53.640 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:53.634+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:53.640 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:53.636+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:53.663 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:53.656+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:53.663 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:53.658+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:53.857 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:53.851+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:54.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:54.373+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:54.394 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:54.387+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:54.566 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:54.559+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:54.641 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:54.636+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:54.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:54.647+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:54.655 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:54.649+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:54.706 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:54.699+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:54.707 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:54.701+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:54.878 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:54.872+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:55.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:55.407+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:55.423 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:55.418+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:55.569 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:55.562+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:55.621 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:55.615+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:55.645 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:55.640+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:55.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:55.644+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:55.750 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:55.743+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:55.753 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:55.747+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:55.926 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:55.920+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:56.415 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:56.406+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:56.470 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:56.463+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:56.565 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:56.558+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:56.660 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:56.653+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:56.661 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:56.656+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:56.671 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:56.666+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:56.770 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:56.760+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:56.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:56.762+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:56.938 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:56.934+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:57.456 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:57.450+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:57.458 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:57.452+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:57.528 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:57.521+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:57.678 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:57.671+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:57.679 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:57.673+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:57.690 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:57.684+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:57.742 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:57.733+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.454975+0000 front 2019-12-09T18:25:42.564863+0000 (oldest deadline 2019-12-09T18:26:28.073201+0000) 2019-12-09T18:26:57.742 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:57.735+0000 1694f700 -1 osd.3 252 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:57.993 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:57.988+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:26:58.432 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:26:58.426+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.392514+0000 front 2019-12-09T18:25:43.325980+0000 (oldest deadline 2019-12-09T18:26:26.138576+0000) 2019-12-09T18:26:58.438 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:26:58.432+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.798026+0000 front 2019-12-09T18:25:40.395011+0000 (oldest deadline 2019-12-09T18:26:23.875002+0000) 2019-12-09T18:26:58.533 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:26:58.526+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:43.039000+0000 front 2019-12-09T18:25:43.381356+0000 (oldest deadline 2019-12-09T18:26:27.627910+0000) 2019-12-09T18:26:58.630 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:26:58.610+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 43sec e252 v252) 2019-12-09T18:26:58.650 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:58.645+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.788463+0000 front 2019-12-09T18:25:41.789022+0000 (oldest deadline 2019-12-09T18:26:24.332557+0000) 2019-12-09T18:26:58.651 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:26:58.646+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:121799 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050419328 gen 1] snapc 0=[] ondisk+write+known_if_redirected e252) 2019-12-09T18:26:58.730 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:26:58.725+0000 1694f700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:41.027898+0000 front 2019-12-09T18:25:40.515168+0000 (oldest deadline 2019-12-09T18:26:24.914176+0000) 2019-12-09T18:26:59.036 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:26:58.911+0000 1694f700 -1 osd.3 253 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:121749 6.8 6.5184c8f8 (undecoded) ondisk+retry+read+known_if_redirected e251) 2019-12-09T18:26:59.039 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:26:59.032+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:25:42.147907+0000 front 2019-12-09T18:25:41.788617+0000 (oldest deadline 2019-12-09T18:26:25.103009+0000) 2019-12-09T18:27:00.054 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:43:56.673 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.060 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:43:56.680 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.070 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:43:56.669 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.074 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:43:56.692 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.074 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:43:56.673 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.077 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:43:56.694 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.289 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:43:56.886 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.461 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:43:57.072 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:00.543 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:43:57.162 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:28.448 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:28.441+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:28.634 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:28.628+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:29.256 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:29.249+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:29.479 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:29.473+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:29.615 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:29.609+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:30.231 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:30.226+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:30.271 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:30.265+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:30.475 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:30.470+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:30.542 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:30.536+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:30.612 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:30.606+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:30.847 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:30.842+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:31.257 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:31.251+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:31.283 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:31.278+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:31.499 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:31.494+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:31.562 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:31.556+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:31.636 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:31.630+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:31.802 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:31.797+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:32.237 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:32.232+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:32.291 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:32.285+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:32.487 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:32.481+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:32.588 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:32.582+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:32.607 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:32.600+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:32.831 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:32.824+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:33.239 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:33.232+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:33.256 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:33.250+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:33.396 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:33.391+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:33.538 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:33.532+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:33.565 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:33.558+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:33.573 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:33.567+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:33.793 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:33.786+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:34.239 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:34.233+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:34.286 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:34.280+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:34.352 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:34.347+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:34.526 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:34.520+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:34.596 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:34.589+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:34.613 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:34.607+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:34.747 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:34.742+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:35.275 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:35.268+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:35.291 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:35.285+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:35.330 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:35.325+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:35.500 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:35.494+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:35.578 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:35.573+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:35.580 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:35.575+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:35.733 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:35.728+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:36.297 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:36.291+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:36.298 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:36.291+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:36.321 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:36.315+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:36.553 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:36.547+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:36.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:36.618+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:36.629 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:36.623+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:36.761 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:36.754+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:37.321 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:37.312+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:37.336 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:37.329+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:37.338 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:37.333+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:37.526 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:37.521+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:37.599 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:37.593+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:37.622 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:37.616+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:37.736 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:37.730+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:38.348 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:38.343+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:38.361 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:38.348+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:38.365 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:38.359+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:38.541 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:38.534+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:38.596 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:38.590+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:38.650 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:38.644+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:38.714 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:38.706+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:39.328 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:39.322+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:39.377 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:39.370+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:39.407 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:39.401+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:39.507 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:39.502+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:39.591 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:39.585+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:39.607 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:39.601+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:39.708 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:39.703+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:40.299 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:40.293+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:40.364 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:40.357+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:40.445 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:40.439+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:40.551 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:40.544+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:40.559 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:40.553+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:40.596 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:40.591+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:40.722 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:40.715+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:41.266 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:41.260+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:41.335 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:41.329+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:41.403 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:41.398+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:41.527 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:41.520+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:41.540 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:41.534+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:41.569 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:41.563+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:41.752 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:41.747+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:42.226 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:42.220+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:42.312 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:42.307+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:42.455 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:42.449+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:42.505 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:42.494+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:42.578 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:42.573+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:42.619 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:42.614+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:42.732 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:42.726+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:43.217 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:43.212+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:43.324 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:43.318+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:43.436 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:43.429+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:43.473 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:43.468+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:43.563 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:43.558+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:43.654 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:43.648+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:43.766 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:43.760+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:44.249 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:44.244+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:44.351 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:44.345+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:44.453 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:44.446+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:44.466 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:44.461+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:44.589 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:44.584+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:44.678 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:44.673+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:44.728 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:44.722+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:45.230 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:45.225+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:45.329 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:45.323+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:45.420 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:45.415+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:45.461 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:45.456+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:45.547 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:45.542+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:45.690 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:45.684+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:45.727 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:45.718+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:46.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:46.200+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:46.346 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:46.336+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:46.412 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:46.407+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:46.498 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:46.493+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:46.527 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:46.518+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:46.674 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:46.669+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:46.757 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:46.751+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:47.180 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:47.174+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:47.369 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:47.363+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:47.441 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:47.434+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:47.497 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:47.493+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:47.550 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:47.546+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:47.693 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:47.688+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:47.726 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:47.719+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:48.202 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:48.195+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:48.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:48.330+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:48.428 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:48.422+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:48.472 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:48.467+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:48.529 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:48.524+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:48.702 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:48.696+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:48.748 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:48.742+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:49.233 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:49.227+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:49.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:49.344+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:49.415 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:49.409+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:49.453 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:49.447+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:49.516 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:49.510+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:49.740 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:49.735+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:49.794 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:49.788+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:50.272 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:50.264+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:50.385 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:50.380+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:50.411 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:50.404+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:50.476 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:50.470+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:50.550 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:50.546+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:50.717 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:50.713+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:50.807 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:50.800+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:51.288 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:51.283+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:51.408 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:51.402+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:51.421 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:51.416+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:51.483 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:51.478+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:51.536 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:51.530+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:51.686 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:51.681+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:51.831 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:51.813+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:52.241 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:52.236+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:52.409 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:52.402+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:52.457 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:52.449+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:52.464 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:52.458+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:52.540 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:52.534+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:52.700 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:52.695+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:52.855 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:52.850+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:53.270 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:53.265+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:53.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:53.389+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:53.448 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:53.441+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:53.448 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:53.441+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:53.508 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:53.502+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:53.693 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:53.687+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:53.869 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:53.862+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:54.251 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:54.245+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:54.375 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:54.370+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:54.485 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:54.478+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:54.485 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:54.479+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:54.562 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:54.555+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:54.702 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:54.696+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:54.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:54.872+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:55.245 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:55.239+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:55.409 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:55.403+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:55.463 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:55.457+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:55.482 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:55.475+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:55.555 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:55.548+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:55.722 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:55.715+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:55.840 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:55.832+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:56.247 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:56.240+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:56.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:56.456+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:56.491 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:56.484+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:56.512 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:56.506+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:56.549 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:56.542+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:56.770 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:56.763+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:56.866 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:56.860+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:57.209 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:57.203+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:57.426 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:57.420+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:57.481 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:57.475+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:57.488 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:57.482+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:57.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:57.543+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:57.793 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:57.785+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:57.916 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:57.905+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:58.193 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:58.185+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:58.386 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:58.380+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:58.469 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:58.460+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:58.495 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:58.488+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:58.553 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:58.546+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:58.760 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:58.753+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:58.875 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:58.870+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:28:59.061 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:28:59.056+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e258 v258) 2019-12-09T18:28:59.181 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:28:59.175+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:28:59.406 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:28:59.400+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:28:59.458 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:28:59.453+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:28:59.467 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:28:59.461+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:28:59.544 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:28:59.538+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:28:59.759 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:28:59.753+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:28:59.834 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:28:59.827+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:00.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:00.198+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:00.404 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:00.399+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:00.441 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:29:00.435+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:43.204517+0000 front 2019-12-09T18:27:43.135130+0000 (oldest deadline 2019-12-09T18:28:28.052568+0000) 2019-12-09T18:29:00.514 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:00.507+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:00.521 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:00.515+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:00.730 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:00.723+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:00.826 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:00.815+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:01.229 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:01.223+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:01.419 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:01.414+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:01.467 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:01.461+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:01.557 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:01.552+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:01.744 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:01.739+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:01.814 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:01.808+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:02.238 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:02.231+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:02.385 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:02.380+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:02.515 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:02.508+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:02.577 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:02.571+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:02.776 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:02.770+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:02.859 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:02.850+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:03.212 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:03.205+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:03.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:03.378+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:03.537 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:03.530+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:03.548 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:03.541+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:03.830 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:03.824+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:03.901 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:03.895+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:04.075 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:29:04.071+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e258 v258) 2019-12-09T18:29:04.215 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:04.209+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:04.421 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:04.416+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:04.503 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:04.497+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:04.589 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:04.577+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:04.851 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:04.843+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:04.931 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:04.925+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:05.193 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:05.187+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:05.447 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:05.442+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:05.536 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:05.530+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:05.637 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:05.631+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:05.841 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:05.832+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:05.980 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:05.971+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:06.249 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:06.242+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:06.419 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:06.413+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:06.546 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:06.538+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:06.687 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:06.681+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:06.884 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:06.879+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:06.966 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:06.960+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:07.218 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:07.211+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:07.468 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:07.462+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:07.577 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:07.571+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:07.682 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:07.677+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:07.853 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:07.848+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:08.011 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:08.004+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:08.217 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:08.211+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.651447+0000 front 2019-12-09T18:27:45.586470+0000 (oldest deadline 2019-12-09T18:28:30.192571+0000) 2019-12-09T18:29:08.441 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:08.436+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:46.283032+0000 front 2019-12-09T18:27:46.095086+0000 (oldest deadline 2019-12-09T18:28:28.926555+0000) 2019-12-09T18:29:08.554 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:08.547+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:08.721 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:29:08.715+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.801627+0000 front 2019-12-09T18:27:47.802779+0000 (oldest deadline 2019-12-09T18:28:28.225819+0000) 2019-12-09T18:29:08.908 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:08.902+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:45.851369+0000 front 2019-12-09T18:27:45.849034+0000 (oldest deadline 2019-12-09T18:28:29.829238+0000) 2019-12-09T18:29:09.006 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:09.000+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.774546+0000 front 2019-12-09T18:27:47.764993+0000 (oldest deadline 2019-12-09T18:28:30.539186+0000) 2019-12-09T18:29:09.136 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:29:09.118+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e258 v258) 2019-12-09T18:29:09.514 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:29:09.509+0000 1694f700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:27:47.697257+0000 front 2019-12-09T18:27:47.643217+0000 (oldest deadline 2019-12-09T18:28:32.648733+0000) 2019-12-09T18:29:52.708 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:52.668+0000 38a7d700 -1 osd.1 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:52.746 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:52.709+0000 34274700 -1 osd.1 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.005 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:52.796+0000 35a77700 -1 osd.1 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.073 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:53.060+0000 39a7f700 -1 osd.0 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.086 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:53.011+0000 114ae700 -1 osd.1 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.098 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:53.094+0000 34274700 -1 osd.4 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.153 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:53.063+0000 39a7f700 -1 osd.0 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.193 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:53.171+0000 34274700 -1 osd.4 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.230 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:53.178+0000 36278700 -1 osd.4 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.241 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.215+0000 34274700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.395 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:53.170+0000 3827c700 -1 osd.0 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.415 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:53.240+0000 36278700 -1 osd.4 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.543 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:29:53.473+0000 35276700 -1 osd.1 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.543 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.403+0000 3a280700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.634 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.414+0000 3a280700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.635 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:53.465+0000 3a280700 -1 osd.0 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.656 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:29:53.501+0000 3927e700 -1 osd.0 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.668 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.441+0000 36a79700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.729 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.540+0000 38a7d700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.746 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.541+0000 33a73700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.829 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:29:53.816+0000 3aa81700 -1 osd.4 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.842 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.577+0000 34a75700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:29:53.873 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:29:53.630+0000 34a75700 -1 osd.6 262 build_incremental_map_msg missing incremental map 262 2019-12-09T18:30:06.221 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:06.212+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:07.134 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:07.128+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:07.259 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:07.252+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:08.167 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:08.163+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:08.307 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:08.300+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:08.399 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:08.393+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:09.172 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:09.166+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:09.215 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:09.209+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:09.293 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:09.287+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:09.365 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:09.358+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:09.548 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:09.542+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:10.155 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:10.142+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:10.197 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:10.192+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:10.295 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:10.287+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:10.335 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:10.330+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:10.536 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:10.530+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:11.133 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:11.127+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:11.213 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:11.208+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:11.254 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:11.248+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:11.355 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:11.349+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:11.586 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:11.580+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:12.101 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:12.094+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:12.197 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:12.187+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:12.263 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:12.253+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:12.353 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:12.342+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:12.584 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:12.578+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:13.101 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:13.098+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:13.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:13.236+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:13.269 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:13.263+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:13.369 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:13.363+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:13.593 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:13.584+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:13.597 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:13.590+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:14.129 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:14.118+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:14.209 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:14.204+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:14.257 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:14.249+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:14.368 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:14.363+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:14.605 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:14.599+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:14.606 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:14.602+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:15.087 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:15.082+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:15.199 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:15.195+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:15.261 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:15.254+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:15.392 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:15.386+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:15.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:15.567+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:15.577 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:15.572+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:16.080 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:16.076+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:16.190 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:16.186+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:16.242 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:16.237+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:16.360 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:16.354+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:16.538 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:16.531+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:16.571 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:16.566+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:17.082 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:17.078+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:17.200 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:17.195+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:17.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:17.241+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:17.380 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:17.374+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:17.532 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:17.522+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:17.543 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:17.539+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:18.060 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:18.056+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:18.185 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:18.180+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:18.293 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:18.283+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:18.382 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:18.377+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:18.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:18.511+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:18.553 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:18.546+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:19.030 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:19.025+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:19.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:19.216+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:19.335 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:19.329+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:19.359 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:19.354+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:19.529 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:19.523+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:19.550 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:19.546+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:20.034 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:20.030+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:20.272 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:20.267+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:20.359 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:20.353+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:20.387 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:20.380+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:20.527 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:20.521+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:20.530 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:20.526+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:21.045 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:21.041+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:21.304 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:21.301+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:21.355 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:21.349+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:21.432 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:21.425+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:21.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:21.493+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:21.500 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:21.495+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:22.027 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:22.021+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:22.269 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:22.264+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:22.313 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:22.307+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:22.413 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:22.407+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:22.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:22.473+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:22.542 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:22.536+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:23.063 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:23.058+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:23.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:23.234+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:23.361 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:23.355+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:23.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:23.444+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:23.495 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:23.489+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:23.569 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:23.563+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:24.057 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:24.052+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:24.176 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:30:24.167+0000 1694f700 -1 osd.3 264 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e262) 2019-12-09T18:30:24.248 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:24.242+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:24.402 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:24.396+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:24.475 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:24.470+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:24.477 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:24.472+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:24.575 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:24.569+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:25.074 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:25.069+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:25.158 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:30:25.147+0000 1694f700 -1 osd.3 264 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e262) 2019-12-09T18:30:25.239 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:25.234+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:25.387 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:25.381+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:25.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:25.464+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:25.519 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:25.513+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:25.563 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:25.558+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:26.054 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:26.050+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:26.145 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:30:26.139+0000 1694f700 -1 osd.3 264 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e262) 2019-12-09T18:30:26.290 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:26.284+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:26.376 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:26.362+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:26.495 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:26.487+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:26.524 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:26.519+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:26.612 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:26.606+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:27.084 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:27.079+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:27.150 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:30:27.144+0000 1694f700 -1 osd.3 264 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e262) 2019-12-09T18:30:27.282 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:27.277+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:27.337 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:27.327+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:27.506 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:27.501+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:27.522 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:27.517+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:27.665 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:27.660+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:28.106 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:28.101+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:28.147 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:30:28.140+0000 1694f700 -1 osd.3 264 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e262) 2019-12-09T18:30:28.278 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:28.267+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:28.301 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:28.295+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:28.552 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:28.545+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:28.556 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:28.549+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:28.687 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:28.681+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:29.129 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:30:29.124+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:26.753298+0000 front 2019-12-09T18:29:27.073421+0000 (oldest deadline 2019-12-09T18:30:06.315777+0000) 2019-12-09T18:30:29.144 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:30:29.139+0000 1694f700 -1 osd.3 264 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e262) 2019-12-09T18:30:29.259 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:30:29.255+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:24.641981+0000 front 2019-12-09T18:29:24.874301+0000 (oldest deadline 2019-12-09T18:30:08.714052+0000) 2019-12-09T18:30:29.270 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:30:29.264+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.246676+0000 front 2019-12-09T18:29:25.268336+0000 (oldest deadline 2019-12-09T18:30:06.092630+0000) 2019-12-09T18:30:29.558 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:30:29.549+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:27.114512+0000 front 2019-12-09T18:29:25.245791+0000 (oldest deadline 2019-12-09T18:30:07.555600+0000) 2019-12-09T18:30:29.561 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:30:29.554+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:30.235786+0000 front 2019-12-09T18:29:27.143066+0000 (oldest deadline 2019-12-09T18:30:13.007017+0000) 2019-12-09T18:30:29.679 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:30:29.669+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:29:25.894813+0000 front 2019-12-09T18:29:25.984053+0000 (oldest deadline 2019-12-09T18:30:09.193560+0000) 2019-12-09T18:30:54.757 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:47:51.373 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:54.761 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:47:51.380 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:54.789 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:47:51.381 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:54.789 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:47:51.398 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:54.790 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:47:51.404 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:54.790 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:47:51.389 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:55.066 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:47:51.679 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:55.089 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:47:51.703 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:55.104 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:47:51.703 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.883 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:48:16.502 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.890 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:48:16.510 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.904 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:48:16.523 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.926 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:48:16.545 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.936 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:48:16.552 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.936 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:48:16.531 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.949 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:48:16.541 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.969 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:48:16.587 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:19.992 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:48:16.591 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:27.275 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:31:27.269+0000 3b282700 -1 osd.6 264 build_incremental_map_msg missing incremental map 262 2019-12-09T18:32:06.199 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:06.192+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:07.189 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:07.184+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:08.155 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:08.147+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:09.187 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:09.182+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:10.145 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:10.139+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:11.154 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:11.149+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:12.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:12.110+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:13.077 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:13.072+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:14.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:14.075+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:15.084 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:15.080+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:16.113 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:16.109+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:17.107 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:17.103+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:18.105 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:18.102+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:19.116 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:19.112+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:20.147 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:20.143+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:21.169 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:21.165+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:22.211 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:22.206+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:23.189 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:23.185+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:24.161 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:24.153+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:25.146 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:25.142+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:26.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:26.130+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:27.123 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:27.119+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:28.104 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:28.099+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:29.066 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:29.061+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:30.098 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:30.095+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:31.082 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:31.078+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:32.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:32.114+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:33.159 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:33.155+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:33.160 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:33.154+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:33.534 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:33.530+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:34.082 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:34.077+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:34.119 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:34.114+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:34.184 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:34.178+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:34.193 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:34.183+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:34.566 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:34.560+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:34.681 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:34.677+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:35.112 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:35.108+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:35.123 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:35.118+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:35.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:35.208+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:35.214 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:35.210+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:35.587 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:35.582+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:35.690 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:35.684+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:36.043 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:36.038+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:36.093 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:36.084+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:36.136 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:36.129+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:36.209 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:36.204+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:36.210 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:36.207+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:36.606 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:36.600+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:36.684 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:36.677+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:37.028 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:37.023+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:37.054 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:37.049+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:37.193 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:37.182+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:37.243 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:37.238+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:37.243 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:37.239+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:37.414 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:37.409+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:37.626 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:37.620+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:37.658 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:37.654+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:38.076 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:38.070+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:38.082 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:38.077+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:38.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:38.209+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:38.238 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:38.232+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:38.239 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:38.234+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:38.409 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:38.398+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:38.619 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:38.613+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:38.639 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:38.632+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:39.068 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:39.063+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:39.079 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:39.073+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:39.225 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:39.220+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:39.267 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:39.263+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:39.268 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:39.264+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:39.455 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:39.449+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:39.588 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:39.583+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:39.603 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:39.597+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:40.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:40.046+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:40.137 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:40.131+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:40.230 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:40.225+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:40.303 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:40.298+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:40.303 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:40.299+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:40.474 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:40.468+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:40.560 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:40.554+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:40.620 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:40.613+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:41.038 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:41.033+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:41.181 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:41.176+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:41.221 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:41.216+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:41.341 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:41.337+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:41.342 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:41.339+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:41.468 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:41.461+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:41.545 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:41.538+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:41.654 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:41.648+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:42.060 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:42.054+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:42.141 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:42.134+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:42.195 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:42.187+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:42.324 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:42.317+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:42.324 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:42.320+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:42.444 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:42.436+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:42.592 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:42.585+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:42.627 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:42.622+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:43.073 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:43.066+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:43.165 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:43.159+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:43.171 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:32:43.165+0000 1694f700 -1 osd.5 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:48.980975+0000 front 2019-12-09T18:31:48.859653+0000 (oldest deadline 2019-12-09T18:32:33.488398+0000) 2019-12-09T18:32:43.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:43.366+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:43.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:43.368+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:43.454 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:43.448+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:43.592 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:43.586+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:43.641 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:43.636+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:44.123 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:44.116+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:44.151 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:44.145+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:44.375 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:44.368+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:44.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:44.371+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:44.415 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:44.409+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:44.579 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:44.573+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:44.674 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:44.669+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:45.129 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:45.124+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:45.202 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:45.196+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:45.367 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:45.361+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:45.367 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:45.362+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:45.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:45.379+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:45.537 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:45.527+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:45.731 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:45.725+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:46.121 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:46.113+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:46.194 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:46.189+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:46.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:46.370+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:46.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:46.371+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:46.399 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:46.394+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:46.564 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:46.557+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:46.693 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:46.685+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:47.089 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:47.076+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:47.186 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:47.180+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:47.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:47.344+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:47.352 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:47.346+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:47.361 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:47.355+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:47.524 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:47.518+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:47.745 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:47.739+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:48.049 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:48.042+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:48.192 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:48.186+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:48.310 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:48.303+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:48.312 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:48.307+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:48.382 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:48.376+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:48.571 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:48.563+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:48.792 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:48.786+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:49.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:49.020+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:49.225 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:49.218+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:49.333 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:49.328+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:49.335 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:49.331+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:49.378 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:49.370+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:49.549 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:49.541+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:49.798 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:49.792+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:50.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:50.044+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:50.218 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:50.213+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:50.377 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:50.371+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:50.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:50.373+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:50.418 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:50.404+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:50.570 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:50.564+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:50.784 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:50.778+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:51.020 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:51.014+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:51.208 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:51.202+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:51.344 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:51.338+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:51.346 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:51.341+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:51.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:51.409+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:51.615 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:51.608+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:51.799 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:51.793+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:52.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:52.047+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:52.221 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:52.216+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:52.309 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:52.299+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:52.310 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:52.302+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:52.458 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:52.453+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:52.655 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:52.649+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:52.804 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:52.798+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:53.076 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:53.070+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:53.263 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:53.258+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:53.355 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:53.348+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:53.355 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:53.349+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:53.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:53.489+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:53.675 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:53.669+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:53.818 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:53.813+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:54.040 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:54.032+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:54.277 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:54.271+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:54.336 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:54.329+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:54.337 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:54.331+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:54.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:54.485+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:54.721 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:54.713+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:54.800 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:54.795+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:55.048 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:55.041+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:55.252 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:55.247+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:55.359 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:55.353+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:55.359 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:55.354+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:55.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:55.510+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:55.755 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:55.748+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:55.802 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:55.797+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:56.090 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:56.083+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:56.240 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:56.232+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:56.405 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:56.397+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:56.406 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:56.401+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:56.527 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:56.520+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:56.713 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:56.706+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:56.824 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:56.818+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:57.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:57.047+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:57.267 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:57.261+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:57.387 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:57.380+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:57.390 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:57.383+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:57.513 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:57.507+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:57.716 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:57.711+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:57.837 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:57.827+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:58.069 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:58.063+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:58.234 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:58.222+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:58.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:58.396+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:58.408 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:58.399+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:58.542 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:58.534+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:58.765 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:58.759+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:58.802 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:58.794+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:32:59.035 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:32:59.029+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:32:59.252 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:32:59.244+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:32:59.419 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:59.412+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:32:59.420 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:32:59.414+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:32:59.520 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:32:59.513+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:32:59.763 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:32:59.757+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:32:59.774 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:32:59.767+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:00.023 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:00.017+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:00.269 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:00.262+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:00.445 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:00.431+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:00.445 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:00.433+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:00.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:00.476+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:00.785 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:00.778+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:00.823 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:00.817+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:01.030 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:01.024+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:01.278 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:01.272+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:01.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:01.410+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:01.418 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:01.412+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:01.525 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:01.519+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:01.803 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:01.797+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:01.822 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:01.815+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:02.012 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:02.001+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:02.333 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:02.326+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:02.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:02.370+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:02.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:02.373+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:02.564 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:02.558+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:02.767 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:02.760+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:02.870 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:02.863+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:02.972 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:02.966+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:03.337 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:03.330+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:03.337 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:03.331+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:03.338 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:03.332+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:03.572 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:03.560+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:03.776 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:03.770+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:03.899 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:03.893+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:04.023 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:04.017+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:04.304 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:04.298+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:04.356 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:04.348+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:04.356 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:04.351+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:04.556 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:04.549+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:04.792 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:04.786+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:04.935 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:04.929+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:05.019 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:05.012+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:05.280 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:05.274+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:05.297 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:33:05.287+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e271 v271) 2019-12-09T18:33:05.323 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:05.314+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:05.323 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:05.316+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:05.585 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:05.580+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:05.842 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:05.835+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:05.912 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:05.899+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:05.998 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:05.991+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:06.317 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:06.306+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:06.342 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:06.332+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:06.344 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:06.334+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:06.604 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:06.597+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:06.843 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:06.836+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:06.911 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:06.901+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:06.979 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:06.973+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:07.319 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:07.313+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:07.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:07.373+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:07.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:07.375+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:07.566 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:07.560+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:07.814 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:07.808+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:07.947 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:07.941+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:07.985 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:07.977+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:08.332 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:08.327+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:08.377 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:08.366+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:08.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:08.368+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:08.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:08.543+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:08.803 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:08.795+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:08.993 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:08.986+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:09.036 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:09.030+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:09.334 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:09.323+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:09.371 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:09.365+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:09.371 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:09.366+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:09.536 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:09.530+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:09.834 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:09.826+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:10.042 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:10.035+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:10.083 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:10.076+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:10.310 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:33:10.303+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e271 v271) 2019-12-09T18:33:10.337 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:10.332+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:10.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:10.345+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:10.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:10.346+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:10.562 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:10.557+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:10.793 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:10.786+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:11.006 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:11.000+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:11.091 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:11.085+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:11.328 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:11.321+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:11.336 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:11.331+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:11.337 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:11.332+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:11.573 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:11.567+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:11.813 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:11.806+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:12.000 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:11.993+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:12.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:12.052+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:12.289 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:12.284+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:12.335 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:12.328+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:12.339 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:12.333+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:12.576 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:12.571+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:12.861 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:12.854+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:12.960 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:12.954+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:13.068 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:13.061+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:13.295 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:13.289+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:13.384 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:13.378+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:13.385 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:13.379+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:13.623 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:13.618+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:13.831 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:13.824+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:13.971 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:13.966+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:14.093 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:14.086+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:14.341 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:14.336+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:14.372 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:14.365+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:14.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:14.367+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:14.653 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:33:14.648+0000 1694f700 -1 osd.4 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.656815+0000 front 2019-12-09T18:31:50.648292+0000 (oldest deadline 2019-12-09T18:32:36.454226+0000) 2019-12-09T18:33:14.868 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:33:14.860+0000 1694f700 -1 osd.1 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:49.579124+0000 front 2019-12-09T18:31:49.284960+0000 (oldest deadline 2019-12-09T18:32:34.408348+0000) 2019-12-09T18:33:15.025 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:33:15.019+0000 1694f700 -1 osd.7 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.564450+0000 front 2019-12-09T18:31:50.769610+0000 (oldest deadline 2019-12-09T18:32:33.327491+0000) 2019-12-09T18:33:15.139 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:33:15.134+0000 1694f700 -1 osd.2 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.780530+0000 front 2019-12-09T18:31:50.752058+0000 (oldest deadline 2019-12-09T18:32:35.300122+0000) 2019-12-09T18:33:15.316 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:33:15.310+0000 1694f700 -1 osd.6 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:47.240673+0000 front 2019-12-09T18:31:47.219129+0000 (oldest deadline 2019-12-09T18:32:33.113894+0000) 2019-12-09T18:33:15.363 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:33:15.352+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e271 v271) 2019-12-09T18:33:15.416 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:15.410+0000 1694f700 -1 osd.0 271 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:31:50.202518+0000 front 2019-12-09T18:31:50.166680+0000 (oldest deadline 2019-12-09T18:32:34.010434+0000) 2019-12-09T18:33:15.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:15.411+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:16.418 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:16.413+0000 1694f700 -1 osd.0 272 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:17.437 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:17.419+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:18.440 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:18.414+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:19.467 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:19.461+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:20.489 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:20.484+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:21.523 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:21.518+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:22.521 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:22.513+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:23.495 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:23.490+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:24.474 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:24.469+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:25.511 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:25.507+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:26.484 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:26.479+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:27.490 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:27.485+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:28.504 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:28.498+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:29.531 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:29.525+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:30.540 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:30.533+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:31.544 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:31.539+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:32.591 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:32.586+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:33.573 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:33.569+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:34.536 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:34.532+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:35.509 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:35.504+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:36.487 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:36.483+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:37.520 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:37.516+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:38.546 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:38.542+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:39.577 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:39.573+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:40.561 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:40.557+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:41.536 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:33:41.532+0000 1694f700 -1 osd.0 273 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122412 3.35 3:af447837:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.629:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e271) 2019-12-09T18:33:47.383 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:33:47.356+0000 2825c700 -1 reset not still connected to 0x3cf31eb0 2019-12-09T18:34:21.221 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:21.215+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:22.214 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:22.209+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:23.180 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:23.174+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:24.143 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:24.138+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:25.173 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:25.162+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:26.192 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:26.186+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:27.205 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:27.199+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:28.218 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:28.213+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:29.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:29.259+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:30.290 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:30.276+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:31.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:31.242+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:32.214 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:32.209+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:33.247 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:33.241+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:34.238 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:34.232+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:35.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:35.250+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:36.220 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:36.214+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:37.222 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:37.217+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:38.240 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:38.234+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:39.221 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:39.215+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:40.197 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:40.191+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:41.251 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:41.246+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:42.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:42.256+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:43.226 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:43.220+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:44.192 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:44.187+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:45.243 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:45.238+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:46.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:46.258+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:47.272 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:47.264+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:47.728 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:34:47.598+0000 20e25700 -1 RGWWatcher::handle_error cookie 1047330704 err (107) Transport endpoint is not connected 2019-12-09T18:34:48.268 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:48.261+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:49.297 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:49.287+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:50.271 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:50.265+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:50.538 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:50.530+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:50.989 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:51:47.608 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:50.997 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:51:47.616 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.003 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:51:47.622 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.006 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:51:47.625 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.011 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:51:47.610 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.018 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:51:47.617 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.259 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:51:47.878 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.270 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:51:47.889 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.294 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:51.288+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:51.420 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:51:48.018 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:34:51.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:51.545+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:52.252 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:52.245+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:52.523 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:52.518+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:53.275 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:53.271+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:53.537 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:53.531+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:54.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:54.252+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:54.539 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:54.535+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:55.217 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:55.213+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:55.525 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:55.519+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:56.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:56.259+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:34:56.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:56.260+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:56.538 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:56.534+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:56.885 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:34:56.876+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:34:57.031 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:34:57.023+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:34:57.222 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:57.217+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:34:57.226 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:57.221+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:57.518 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:57.513+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:57.805 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:34:57.800+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:34:57.857 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:34:57.851+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:34:58.059 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:34:58.053+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:34:58.264 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:58.257+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:34:58.267 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:58.259+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:58.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:58.509+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:58.821 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:34:58.815+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:34:58.858 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:34:58.854+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:34:59.089 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:34:59.084+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:34:59.271 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:59.265+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:34:59.275 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:34:59.270+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:34:59.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:34:59.543+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:34:59.819 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:34:59.814+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:34:59.842 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:34:59.836+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:34:59.875 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:34:59.869+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:00.072 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:00.067+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:00.249 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:00.243+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:00.250 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:00.245+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:00.529 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:00.523+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:00.529 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:00.524+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:00.807 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:00.802+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:00.847 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:00.842+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:00.908 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:00.903+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:01.081 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:01.073+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:01.169 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:01.161+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:01.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:01.248+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:01.257 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:01.252+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:01.544 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:01.538+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:01.545 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:01.540+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:01.776 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:01.771+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:01.868 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:01.862+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:01.872 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:01.867+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:02.066 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:02.059+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:02.203 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:02.197+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:02.280 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:02.276+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:02.283 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:02.279+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:02.591 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:02.583+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:02.592 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:02.585+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:02.736 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:02.730+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:02.912 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:02.908+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:02.920 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:02.915+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:03.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:03.110+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:03.182 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:03.178+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:03.303 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:03.297+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:03.307 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:03.298+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:03.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:03.567+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:03.575 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:03.569+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:03.779 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:03.774+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:03.879 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:03.873+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:03.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:03.958+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:04.121 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:04.114+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:04.154 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:04.150+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:04.353 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:04.348+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:04.356 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:04.351+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:04.565 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:04.559+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:04.568 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:04.563+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:04.763 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:04.754+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:04.877 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:04.871+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:04.932 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:04.927+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:05.173 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:05.168+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:05.198 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:05.193+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:05.328 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:05.323+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:05.329 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:05.325+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:05.592 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:05.586+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:05.593 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:05.588+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:05.730 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:05.723+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:05.905 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:05.899+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:05.983 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:05.977+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:06.162 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:06.156+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:06.196 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:06.191+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:06.317 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:06.312+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:06.317 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:06.313+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:06.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:06.594+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:06.601 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:06.596+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:06.737 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:06.731+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:06.900 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:06.893+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:07.015 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:07.009+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:07.142 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:07.136+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:07.156 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:07.151+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:07.281 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:07.276+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:07.284 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:07.278+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:07.620 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:07.613+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:07.620 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:07.615+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:07.743 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:07.737+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:07.924 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:07.911+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:08.019 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:08.013+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:08.178 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:08.172+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:08.200 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:08.195+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:08.282 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:08.277+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:08.286 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:08.282+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:08.618 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:08.612+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:08.621 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:08.617+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:08.781 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:08.774+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:08.964 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:08.958+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:08.989 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:08.984+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:09.152 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:09.147+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:09.241 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:09.236+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:09.242 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:09.237+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:09.242 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:09.238+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:09.619 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:09.613+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:09.620 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:09.615+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:09.750 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:09.745+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:09.921 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:09.915+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:10.010 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:10.004+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:10.192 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:10.187+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:10.225 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:10.219+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:10.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:10.218+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:10.226 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:10.221+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:10.668 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:10.662+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:10.669 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:10.664+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:10.713 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:10.709+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:10.895 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:10.890+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:11.046 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:11.040+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:11.218 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:11.212+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:11.218 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:11.212+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:11.219 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:11.213+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:11.248 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:11.243+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:11.639 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:11.633+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:11.639 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:11.635+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:11.706 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:11.700+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:11.916 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:11.911+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:12.003 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:11.998+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:12.176 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:12.172+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:12.261 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:12.257+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:12.266 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:12.260+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:12.299 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:12.295+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:12.618 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:12.611+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:12.618 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:12.613+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:12.703 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:12.698+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:12.946 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:12.940+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:13.030 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:13.025+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:13.166 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:13.161+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:13.261 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:13.256+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:13.262 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:13.258+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:13.305 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:13.297+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:13.663 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:13.657+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:13.663 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:13.658+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:13.702 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:13.696+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:13.937 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:13.932+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:13.998 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:13.993+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:14.186 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:14.180+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:14.277 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:14.272+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:14.295 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:14.290+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:14.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:14.291+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:14.662 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:14.657+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:14.664 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:14.660+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:14.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:14.667+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:14.897 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:14.892+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:15.046 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:15.041+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:15.193 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:15.188+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:15.311 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:15.306+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:15.314 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:15.310+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:15.332 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:15.327+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:15.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:15.648+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:15.657 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:15.653+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:15.684 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:15.679+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:15.923 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:15.918+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:16.007 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:16.003+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:16.241 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:16.231+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:16.287 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:16.283+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:16.347 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:16.343+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:16.348 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:16.345+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:16.640 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:16.635+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:16.658 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:16.651+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:16.661 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:16.653+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:16.882 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:16.878+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:16.990 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:16.985+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:17.260 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:17.255+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:17.311 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:17.305+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:17.381 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:17.373+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:17.383 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:17.379+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:17.602 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:17.597+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:17.656 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:17.650+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:17.657 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:17.651+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:17.880 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:17.875+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:17.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:17.956+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:18.278 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:18.273+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:18.281 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:18.275+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:18.425 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:18.419+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:18.426 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:18.421+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:18.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:18.629+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:18.638 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:18.634+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:18.644 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:18.639+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:18.863 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:18.857+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:18.926 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:18.920+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:19.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:19.236+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:19.295 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:19.290+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:19.397 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:19.388+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:19.397 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:19.390+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:19.626 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:19.621+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:19.669 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:19.663+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:19.670 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:19.665+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:19.848 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:19.842+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:19.915 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:19.909+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:20.234 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:20.228+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:20.281 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:20.275+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:20.428 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:20.422+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:20.430 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:20.426+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:20.639 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:20.635+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:20.640 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:20.635+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:20.652 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:20.645+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:20.817 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:20.811+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:20.934 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:20.929+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:21.264 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:21.258+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:21.312 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:21.307+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:21.393 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:21.388+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:21.395 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:21.390+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:21.598 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:21.593+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:21.602 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:21.598+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:21.702 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:21.696+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:21.786 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:21.780+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:21.959 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:21.954+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:22.240 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:22.232+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:22.309 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:22.303+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:22.390 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:22.384+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:22.393 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:22.388+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:22.602 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:22.595+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:22.603 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:22.598+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:22.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:22.681+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:22.790 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:22.784+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:23.001 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:22.996+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:23.274 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:23.269+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:23.275 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:23.269+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:23.444 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:23.437+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:23.444 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:23.438+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:23.629 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:23.617+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:23.629 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:23.621+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:23.646 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:23.639+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:23.770 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:23.764+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:24.052 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:24.046+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:24.248 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:24.242+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:24.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:24.299+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:24.420 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:24.414+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:24.422 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:24.415+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:24.584 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:24.577+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:24.585 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:24.580+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:24.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:24.664+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:24.776 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:24.770+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:25.026 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:25.020+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:25.300 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:25.290+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:25.300 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:25.292+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:25.432 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:25.425+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:25.433 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:25.427+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:25.610 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:25.604+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:25.611 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:25.606+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:25.719 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:25.713+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:25.827 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:25.821+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:26.077 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:26.072+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:26.277 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:26.272+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:26.289 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:26.283+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:26.477 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:26.472+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:26.478 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:26.472+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:26.640 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:26.635+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:26.644 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:26.639+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:26.748 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:26.738+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:26.797 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:26.782+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:27.098 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:27.092+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:27.333 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:27.326+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:27.339 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:27.333+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:27.484 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:27.478+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:27.487 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:27.481+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:27.596 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:27.593+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:27.599 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:27.596+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:27.731 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:27.724+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:27.757 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:27.744+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:28.128 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:28.123+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:28.321 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:28.316+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:28.349 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:28.343+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:28.500 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:28.494+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:28.503 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:28.498+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:28.583 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:28.579+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:28.587 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:28.583+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:28.716 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:28.710+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:28.740 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:28.736+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:29.109 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:29.105+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:29.279 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:29.271+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:29.353 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:29.347+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:29.520 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:29.515+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:29.523 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:29.518+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:29.541 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:29.536+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:29.543 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:29.539+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:29.751 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:29.744+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:29.771 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:29.767+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:30.126 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:30.122+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:30.294 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:30.288+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:30.364 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:30.358+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:30.503 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:30.495+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:30.504 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:30.497+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:30.579 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:30.574+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:30.580 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:30.576+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:30.766 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:30.759+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:30.800 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:30.795+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:30.941 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:35:30.932+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e277 v277) 2019-12-09T18:35:31.120 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:31.112+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:31.335 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:31.329+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:31.403 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:31.398+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:31.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:31.541+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:31.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:31.545+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:31.549 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:31.543+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:31.550 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:31.544+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:31.816 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:31.809+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:34:12.821006+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:31.817 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:31.811+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:17.001279+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:32.106 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:32.102+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:32.380 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:32.374+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.165696+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:32.431 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:32.424+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:34:16.403562+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:32.524 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:32.520+0000 1694f700 -1 osd.4 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:16.536932+0000 front 2019-12-09T18:34:16.999768+0000 (oldest deadline 2019-12-09T18:34:59.697128+0000) 2019-12-09T18:35:32.525 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:32.521+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:32.547 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:32.541+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:34:16.280010+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:32.547 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:32.542+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:32.797 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:32.792+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:32.777653+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:32.843 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:32.837+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:35:32.634449+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:33.145 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:33.141+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:33.373 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:33.367+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:33.357460+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:33.437 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:33.431+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:35:33.090907+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:33.511 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:33.507+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:33.578 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:33.573+0000 1694f700 -1 osd.3 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.730644+0000 front 2019-12-09T18:35:32.665104+0000 (oldest deadline 2019-12-09T18:34:56.136959+0000) 2019-12-09T18:35:33.581 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:33.576+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:33.780 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:33.775+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:32.777653+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:33.837 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:33.828+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:35:32.634449+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:34.163 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:35:34.160+0000 1694f700 -1 osd.7 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.709881+0000 front 2019-12-09T18:34:22.708363+0000 (oldest deadline 2019-12-09T18:34:59.784818+0000) 2019-12-09T18:35:34.339 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:34.333+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:33.357460+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:34.373 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:35:34.344+0000 1694f700 -1 osd.6 275 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4367.0:130649 7.7 7:e9187ab8:reshard::reshard.0000000000:head [call lock.lock in=70b] snapc 0=[] RETRY=36 ondisk+retry+write+known_if_redirected e272) 2019-12-09T18:35:34.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:34.411+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:35:33.090907+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:34.555 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:34.550+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:34.618 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:34.613+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:34.776 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:34.772+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:32.777653+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:34.820 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:34.813+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:35:32.634449+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:35.334 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:35:35.328+0000 1694f700 -1 osd.2 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:34.612174+0000 front 2019-12-09T18:34:22.744017+0000 (oldest deadline 2019-12-09T18:35:00.998988+0000) 2019-12-09T18:35:35.377 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:35.371+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:35:33.090907+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:35.569 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:35.565+0000 1694f700 -1 osd.4 277 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:35.581 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:35:35.575+0000 1694f700 -1 osd.3 277 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:122608 2.2fs0 2.89604c2f (undecoded) ondisk+retry+write+known_if_redirected e275) 2019-12-09T18:35:35.737 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:35:35.732+0000 1694f700 -1 osd.5 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:35:32.777653+0000 front 2019-12-09T18:34:22.707078+0000 (oldest deadline 2019-12-09T18:34:56.196895+0000) 2019-12-09T18:35:35.796 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:35:35.789+0000 1694f700 -1 osd.1 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:13.052352+0000 front 2019-12-09T18:35:32.634449+0000 (oldest deadline 2019-12-09T18:34:57.493609+0000) 2019-12-09T18:35:36.021 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:35:36.013+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e277 v277) 2019-12-09T18:35:36.407 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:35:36.400+0000 1694f700 -1 osd.0 277 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:34:22.397706+0000 front 2019-12-09T18:35:33.090907+0000 (oldest deadline 2019-12-09T18:34:56.016302+0000) 2019-12-09T18:35:36.704 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:35:36.601+0000 1694f700 -1 osd.6 277 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4367.0:130649 7.7 7:e9187ab8:reshard::reshard.0000000000:head [call lock.lock in=70b] snapc 0=[] RETRY=36 ondisk+retry+write+known_if_redirected e272) 2019-12-09T18:35:36.711 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:36.684+0000 1694f700 -1 osd.4 278 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:35:37.789 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:35:37.769+0000 1694f700 -1 osd.4 279 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:122672 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 4] snapc 0=[] ondisk+write+known_if_redirected e277) 2019-12-09T18:37:00.552 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:00.543+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:01.392 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:01.387+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:01.523 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:01.518+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:01.597 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:01.588+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:02.053 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:02.047+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:02.438 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:02.430+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:02.494 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:02.484+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:02.558 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:02.552+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:02.985 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:02.979+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:03.062 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:03.055+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:03.296 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:03.291+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:03.332 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:03.325+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:03.434 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:03.428+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:03.458 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:03.453+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:03.592 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:03.585+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:03.969 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:03.958+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:04.073 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:04.064+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:04.301 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:04.293+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:04.302 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:04.293+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:04.453 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:04.448+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:04.493 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:04.488+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:04.616 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:04.605+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:04.934 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:04.927+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:05.116 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:05.110+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:05.270 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:05.265+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:05.332 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:05.327+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:05.487 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:05.482+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:05.543 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:05.537+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:05.660 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:05.654+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:05.967 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:05.960+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:06.117 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:06.106+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:06.258 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:06.243+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:06.365 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:06.360+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:06.489 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:06.485+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:06.512 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:06.507+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:06.685 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:06.678+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:06.946 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:06.938+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:07.103 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:07.096+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:07.217 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:07.208+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:07.384 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:07.378+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:07.479 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:07.473+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:07.519 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:07.510+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:07.693 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:07.683+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:07.976 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:07.969+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:08.126 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:08.121+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:08.238 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:08.229+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:08.381 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:08.375+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:08.496 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:08.490+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:08.525 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:08.519+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:08.716 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:08.707+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:09.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:09.000+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:09.113 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:09.104+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:09.281 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:09.274+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:09.409 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:09.402+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:09.499 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:09.493+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:09.577 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:09.569+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:09.760 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:09.752+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:10.061 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:10.052+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:10.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:10.103+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:10.296 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:10.290+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:10.365 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:10.359+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:10.457 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:10.451+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:10.551 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:10.545+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:10.761 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:10.753+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:11.033 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:11.027+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:11.157 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:11.150+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:11.272 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:11.265+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:11.352 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:11.345+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:11.453 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:11.446+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:11.522 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:11.514+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:11.739 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:11.730+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:11.850 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:54:08.469 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.910 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:54:08.515 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.941 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:54:08.560 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.976 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:54:08.574 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.977 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:54:08.596 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.984 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:54:08.583 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:12.026 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:12.020+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:12.089 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:54:08.708 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:12.124 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:12.119+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:12.143 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:54:08.742 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:12.151 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:54:08.770 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:12.315 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:12.303+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:12.397 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:12.390+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:12.503 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:12.493+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:12.531 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:12.525+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:12.698 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:12.692+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:12.983 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:12.977+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:13.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:13.112+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:13.303 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:13.298+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:13.449 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:13.443+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:13.556 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:13.548+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:13.557 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:13.549+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:13.695 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:13.689+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:13.989 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:13.981+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:14.098 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:14.091+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:14.319 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:14.312+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:14.452 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:14.443+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:14.528 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:14.523+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:14.574 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:14.567+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:14.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:14.665+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:14.968 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:14.962+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:15.055 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:15.048+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:15.355 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:15.350+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:15.450 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:15.444+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:15.547 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:15.541+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:15.557 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:15.549+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:15.686 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:15.679+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:16.008 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:16.001+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:16.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:16.089+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:16.331 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:16.323+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:16.413 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:16.407+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:16.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:16.542+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:16.591 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:16.585+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:16.709 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:16.702+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:16.998 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:16.992+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:17.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:17.076+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:17.319 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:17.313+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:17.421 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:17.415+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:17.553 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:17.547+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:17.582 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:17.577+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:17.697 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:17.690+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:18.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:18.021+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:18.084 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:18.076+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:18.350 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:18.346+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:18.451 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:18.447+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:18.557 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:18.549+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:18.639 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:18.631+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:18.671 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:18.663+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:19.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:19.020+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:19.041 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:19.034+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:19.377 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:19.371+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:19.429 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:19.423+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:19.551 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:19.546+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:19.598 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:19.592+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:19.637 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:19.630+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:20.042 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:20.035+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:20.092 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:20.085+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:20.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:20.329+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:20.470 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:20.465+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:20.564 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:20.557+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:20.617 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:20.611+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:20.651 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:20.643+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:21.079 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:21.073+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:21.122 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:21.114+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:21.299 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:21.293+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:21.490 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:21.485+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:21.570 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:21.563+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:21.663 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:21.658+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:21.680 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:21.673+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:22.096 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:22.089+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:22.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:22.107+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:22.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:22.268+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:22.479 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:22.474+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:22.529 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:22.520+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:22.662 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:22.655+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:22.729 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:22.724+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:23.100 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:23.091+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:23.166 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:23.157+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:23.280 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:23.274+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:23.455 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:23.443+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:23.556 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:23.551+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:23.655 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:23.647+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:23.695 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:23.688+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:24.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:24.112+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:24.142 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:24.136+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:24.291 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:24.284+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:24.425 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:24.419+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:24.549 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:24.543+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:24.639 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:24.632+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:24.720 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:24.713+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:25.148 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:25.141+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:25.156 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:25.150+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:25.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:25.276+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:25.456 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:25.450+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:25.584 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:25.578+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:25.624 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:25.617+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:25.725 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:25.719+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:26.135 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:26.129+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:26.193 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:26.185+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:26.303 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:26.297+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:26.474 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:26.466+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:26.563 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:26.557+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:26.645 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:26.638+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:26.715 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:26.708+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:27.159 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:27.151+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:27.242 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:27.236+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:27.305 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:27.299+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:27.487 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:27.481+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:27.587 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:27.581+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:27.600 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:27.594+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:27.707 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:27.700+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:28.187 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:28.180+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:28.283 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:28.276+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:28.325 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:28.315+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:28.500 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:28.494+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:28.565 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:28.559+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:28.605 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:28.598+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:28.712 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:28.704+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:29.200 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:29.195+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:29.256 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:29.249+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:29.283 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:29.277+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:29.460 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:29.454+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:29.563 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:29.557+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:29.610 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:29.604+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:29.736 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:29.731+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:30.214 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:30.208+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:30.307 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:30.302+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:30.315 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:30.309+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:30.423 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:30.418+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:30.556 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:30.550+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:30.570 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:30.564+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:30.729 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:30.722+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:31.214 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:31.206+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:31.339 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:31.329+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:31.353 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:31.347+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:31.407 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:31.402+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:31.529 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:31.523+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:31.560 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:31.553+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:31.766 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:31.762+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:32.228 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:32.222+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:32.300 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:32.293+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:32.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:32.368+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:32.408 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:32.389+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:32.512 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:32.500+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:32.595 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:32.590+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:32.727 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:32.721+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:33.190 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:33.185+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:33.328 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:33.323+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:33.332 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:33.326+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:33.407 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:33.401+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:33.532 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:33.526+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:33.646 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:33.641+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:33.697 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:33.691+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:34.166 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:34.160+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:34.295 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:34.289+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:34.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:34.376+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:34.430 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:34.424+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:34.571 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:34.565+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:34.689 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:34.683+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:34.702 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:34.696+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:35.151 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:35.146+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:35.346 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:35.341+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:35.426 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:35.420+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:35.494 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:35.485+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:35.578 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:35.572+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:35.688 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:35.681+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:35.701 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:35.695+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:36.144 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:36.139+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:36.320 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:36.313+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:36.454 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:36.448+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:36.460 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:36.455+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:36.614 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:36.608+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:36.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:36.667+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:36.696 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:36.691+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:36.880 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:37:36.875+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e281 v281) 2019-12-09T18:37:37.110 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:37.106+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:37.331 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:37.325+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:37.431 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:37.425+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:37.465 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:37.459+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:37.587 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:37.581+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:37.671 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:37.664+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:37.702 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:37.695+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:38.161 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:38.151+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:38.355 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:38.350+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:38.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:38.472+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:38.501 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:38.495+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:38.626 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:38.619+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:38.721 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:38.715+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:38.726 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:38.719+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:39.117 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:39.107+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:39.357 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:39.352+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:39.458 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:39.447+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:39.512 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:39.506+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:39.645 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:39.639+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:39.686 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:39.680+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:39.756 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:39.751+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:40.162 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:40.158+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:40.389 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:40.383+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:40.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:40.412+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:40.537 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:40.528+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:40.683 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:40.677+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:40.736 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:40.731+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:40.737 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:40.732+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:41.217 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:37:41.212+0000 1694f700 -1 osd.2 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.456905+0000 front 2019-12-09T18:36:20.143194+0000 (oldest deadline 2019-12-09T18:37:02.300657+0000) 2019-12-09T18:37:41.397 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:37:41.387+0000 1694f700 -1 osd.0 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.193689+0000 front 2019-12-09T18:36:17.928462+0000 (oldest deadline 2019-12-09T18:37:01.900943+0000) 2019-12-09T18:37:41.467 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:37:41.458+0000 1694f700 -1 osd.4 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:18.847323+0000 front 2019-12-09T18:36:18.872788+0000 (oldest deadline 2019-12-09T18:37:02.316251+0000) 2019-12-09T18:37:41.529 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:37:41.522+0000 1694f700 -1 osd.7 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:20.155375+0000 front 2019-12-09T18:36:20.157477+0000 (oldest deadline 2019-12-09T18:37:02.759924+0000) 2019-12-09T18:37:41.648 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:37:41.642+0000 1694f700 -1 osd.5 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:19.970050+0000 front 2019-12-09T18:36:19.899617+0000 (oldest deadline 2019-12-09T18:37:00.951636+0000) 2019-12-09T18:37:41.702 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:37:41.696+0000 1694f700 -1 osd.1 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:17.809013+0000 front 2019-12-09T18:36:19.260903+0000 (oldest deadline 2019-12-09T18:37:00.536159+0000) 2019-12-09T18:37:41.794 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:37:41.786+0000 1694f700 -1 osd.6 281 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:36:16.048280+0000 front 2019-12-09T18:36:17.261046+0000 (oldest deadline 2019-12-09T18:37:01.019443+0000) 2019-12-09T18:37:41.945 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:37:41.935+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e281 v281) 2019-12-09T18:38:21.163 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:38:21.040+0000 3827c700 -1 osd.0 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.166 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:38:21.134+0000 3827c700 -1 osd.5 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:38:21.117+0000 35276700 -1 osd.0 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.327 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:38:21.164+0000 34a75700 -1 osd.0 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.332 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:38:21.286+0000 3aa81700 -1 osd.5 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.377 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:38:21.326+0000 34274700 -1 osd.5 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.395 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:38:21.175+0000 3827c700 -1 osd.0 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.498 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:38:21.238+0000 35a77700 -1 osd.0 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:21.498 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:38:21.445+0000 38a7d700 -1 osd.5 285 build_incremental_map_msg missing incremental map 285 2019-12-09T18:38:45.514 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:55:42.133 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.523 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:55:42.141 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.534 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:55:42.132 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.535 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:55:42.154 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.544 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:55:42.139 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.545 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:55:42.158 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.637 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:55:42.256 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.657 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:55:42.275 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:45.697 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:55:42.295 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:52.957 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:52.953+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:53.993 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:53.988+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:55.009 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:55.004+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:55.985 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:55.980+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:57.035 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:57.027+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:57.990 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:57.985+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:58.986 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:58.982+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:38:59.945 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:38:59.939+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:00.998 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:00.994+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:01.954 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:01.949+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:02.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:02.990+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:04.046 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:04.034+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:05.017 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:05.012+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:05.997 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:05.992+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:06.993 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:06.988+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:08.040 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:08.035+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:09.013 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:09.008+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:10.026 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:10.021+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:11.015 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:11.010+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:12.026 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:12.021+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:13.022 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:13.009+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:14.031 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:14.026+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:15.046 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:15.042+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:16.020 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:16.016+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:17.038 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:17.033+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:18.036 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:18.030+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:19.055 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:19.049+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:20.026 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:20.020+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:21.027 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:21.021+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:21.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:21.989+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:22.980 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:22.974+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:24.005 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:23.999+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:24.962 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:24.956+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:25.002 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:24.997+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:25.604 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:25.598+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:25.633 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:25.627+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:25.639 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:25.630+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:25.982 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:25.976+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:26.002 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:25.995+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:26.601 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:26.589+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:26.647 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:26.638+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:26.653 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:26.647+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:26.853 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:26.846+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:26.978 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:26.973+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:27.029 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:27.021+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:27.571 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:27.565+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:27.611 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:27.605+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:27.683 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:27.677+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:27.753 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:27.748+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:27.808 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:27.802+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:27.983 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:27.976+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:28.058 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:28.053+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:28.619 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:28.614+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:28.662 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:28.657+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:28.681 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:28.676+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:28.788 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:28.782+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:28.839 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:28.832+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:29.021 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:29.016+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:29.080 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:29.074+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:29.082 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:29.077+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:29.641 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:29.636+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:29.664 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:29.659+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:29.731 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:29.726+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:29.821 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:29.816+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:29.882 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:29.876+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:29.993 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:29.986+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:30.064 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:30.058+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:30.067 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:30.062+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:30.635 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:30.629+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:30.658 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:30.652+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:30.689 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:30.683+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:30.866 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:30.861+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:30.875 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:30.868+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:31.035 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:31.030+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:31.064 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:31.058+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:31.070 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:31.061+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:31.622 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:31.616+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:31.709 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:31.702+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:31.733 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:31.728+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:31.879 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:31.874+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:31.892 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:31.885+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:32.091 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:32.085+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:32.103 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:32.097+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:32.104 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:32.100+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:32.657 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:32.652+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:32.695 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:32.689+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:32.718 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:32.711+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:32.836 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:32.829+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:32.913 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:32.906+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:33.047 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:33.042+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:33.065 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:33.059+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:33.066 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:33.061+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:33.659 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:33.652+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:33.690 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:33.683+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:33.739 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:33.733+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:33.812 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:33.807+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:33.912 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:33.905+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:34.040 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:34.034+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:34.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:34.035+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:34.053 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:34.047+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:34.695 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:34.688+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:34.705 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:34.700+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:34.767 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:34.761+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:34.816 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:34.811+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:34.929 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:34.923+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:35.055 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:35.051+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:35.087 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:35.079+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:35.087 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:35.080+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:35.670 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:35.664+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:35.729 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:35.724+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:35.769 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:35.764+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:35.785 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:35.779+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:35.915 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:35.909+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:36.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:36.045+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:36.086 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:36.076+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:36.086 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:36.078+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:36.644 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:36.638+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:36.700 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:36.695+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:36.754 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:36.748+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:36.767 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:36.757+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:36.929 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:36.923+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:37.008 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:37.003+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:37.072 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:37.065+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:37.075 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:37.070+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:37.684 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:37.677+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:37.707 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:37.702+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:37.733 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:37.728+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:37.787 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:37.780+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:37.905 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:37.899+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:38.036 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:38.030+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:38.124 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:38.117+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:38.125 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:38.119+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:38.698 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:38.691+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:38.728 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:38.723+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:38.749 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:38.744+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:38.789 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:38.782+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:38.873 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:38.863+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:39.019 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:39.013+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:39.108 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:39.101+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:39.108 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:39.103+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:39.693 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:39.687+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:39.701 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:39.695+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:39.755 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:39.749+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:39.777 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:39.770+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:39.885 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:39.874+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:40.008 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:40.002+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:40.094 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:40.087+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:40.097 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:40.091+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:40.673 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:40.667+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:40.695 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:40.690+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:40.757 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:40.751+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:40.827 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:40.820+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:40.880 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:40.873+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:41.019 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:41.013+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:41.119 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:41.112+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:41.120 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:41.114+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:41.687 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:41.681+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:41.739 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:41.732+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:41.784 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:41.778+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:41.811 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:41.806+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:41.920 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:41.914+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:42.017 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:42.011+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:42.153 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:42.146+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:42.153 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:42.147+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:42.590 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:56:39.208 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.604 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:56:39.223 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.614 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:56:39.233 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.620 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:56:39.237 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.652 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:56:39.248 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.656 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:56:39.254 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.679 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:56:39.298 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.693 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:42.687+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:42.697 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:42.692+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:42.710 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:56:39.327 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.742 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:42.736+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:42.774 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:56:39.372 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:42.838 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:42.832+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:42.915 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:42.908+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:43.036 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:43.032+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:43.122 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:43.113+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:43.123 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:43.114+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:43.689 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:43.682+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:43.695 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:43.690+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:43.794 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:43.788+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:43.861 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:43.855+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:43.904 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:43.898+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:44.090 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:44.084+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:44.091 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:44.086+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:44.093 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:44.088+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:44.733 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:44.727+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:44.743 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:44.738+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:44.785 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:44.771+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:44.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:44.874+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:44.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:44.937+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:45.063 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:45.057+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:45.109 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:45.102+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:45.109 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:45.104+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:45.723 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:45.718+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:45.735 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:45.730+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:45.755 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:45.749+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:45.873 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:45.867+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:45.913 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:45.907+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:46.100 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:46.094+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:46.121 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:46.114+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:46.122 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:46.116+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:46.719 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:46.713+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:46.752 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:46.746+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:46.772 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:46.767+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:46.866 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:46.860+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:46.952 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:46.945+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:47.100 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:47.090+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:47.100 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:47.092+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:47.129 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:47.124+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:47.748 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:47.741+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:47.750 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:47.744+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:47.769 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:47.764+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:47.913 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:47.902+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:47.928 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:47.920+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:48.088 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:48.081+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:48.088 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:48.083+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:48.115 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:48.110+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:48.731 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:48.725+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:48.748 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:48.741+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:48.814 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:48.809+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:48.905 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:48.899+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:48.934 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:48.928+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:49.071 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:49.064+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:49.073 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:49.066+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:49.109 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:49.103+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:49.726 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:49.721+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:49.798 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:49.792+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:49.804 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:49.798+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:49.883 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:49.878+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:49.987 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:49.981+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:50.102 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:50.095+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:50.113 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:50.102+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:50.114 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:50.101+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:50.740 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:50.735+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:50.773 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:50.767+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:50.811 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:50.806+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:50.895 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:50.889+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:50.993 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:50.986+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:51.075 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:51.069+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:51.118 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:51.110+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:51.119 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:51.112+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:51.724 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:51.719+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:51.815 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:51.808+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:51.827 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:51.821+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:51.946 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:51.940+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:51.995 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:51.988+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:52.081 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:52.071+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:52.120 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:52.113+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:52.121 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:52.115+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:52.736 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:52.730+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:52.847 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:52.842+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:52.859 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:52.852+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:52.999 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:52.993+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:53.026 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:53.020+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:53.089 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:53.079+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:53.094 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:53.086+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:53.094 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:53.088+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:53.771 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:53.766+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:53.840 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:53.834+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:53.876 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:53.872+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:53.988 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:53.982+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:54.039 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:54.032+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:54.061 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:54.054+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:54.062 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:54.057+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:54.062 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:54.056+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:54.752 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:54.746+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:54.837 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:54.831+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:54.867 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:54.860+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:54.979 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:54.971+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:55.010 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:55.004+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:55.045 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:55.038+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:55.046 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:55.040+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:55.116 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:55.110+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:55.727 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:55.722+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:55.843 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:55.835+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:55.853 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:55.848+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:55.932 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:55.926+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:56.027 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:56.019+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:56.027 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:56.021+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:56.049 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:56.043+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:56.128 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:56.122+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:56.765 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:39:56.761+0000 1694f700 -1 osd.5 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.494260+0000 front 2019-12-09T18:38:44.319150+0000 (oldest deadline 2019-12-09T18:39:25.395925+0000) 2019-12-09T18:39:56.857 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:56.851+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:39:56.896 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:39:56.884+0000 1694f700 -1 osd.2 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.486692+0000 front 2019-12-09T18:38:42.701239+0000 (oldest deadline 2019-12-09T18:39:25.421080+0000) 2019-12-09T18:39:56.974 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:39:56.967+0000 1694f700 -1 osd.0 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.460888+0000 front 2019-12-09T18:38:46.045244+0000 (oldest deadline 2019-12-09T18:39:24.802729+0000) 2019-12-09T18:39:57.008 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:57.001+0000 1694f700 -1 osd.3 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:44.070101+0000 front 2019-12-09T18:38:44.121554+0000 (oldest deadline 2019-12-09T18:39:28.379337+0000) 2019-12-09T18:39:57.008 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:39:57.002+0000 1694f700 -1 osd.3 288 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e285) 2019-12-09T18:39:57.020 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:39:57.014+0000 1694f700 -1 osd.1 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.575125+0000 front 2019-12-09T18:38:44.059330+0000 (oldest deadline 2019-12-09T18:39:26.030527+0000) 2019-12-09T18:39:57.095 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:39:57.091+0000 1694f700 -1 osd.4 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:42.896032+0000 front 2019-12-09T18:38:42.898398+0000 (oldest deadline 2019-12-09T18:39:24.726106+0000) 2019-12-09T18:39:57.651 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:39:57.641+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e288 v288) 2019-12-09T18:39:57.853 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:39:57.849+0000 1694f700 -1 osd.7 288 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:38:45.992559+0000 front 2019-12-09T18:38:45.494761+0000 (oldest deadline 2019-12-09T18:39:27.694059+0000) 2019-12-09T18:41:08.137 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:08.131+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:08.890 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:08.882+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:09.137 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:09.130+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:09.907 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:09.901+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:10.032 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:10.024+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:10.045 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:10.039+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:10.171 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:10.165+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:10.200 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:10.194+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:10.344 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:10.338+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:10.863 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:10.856+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:10.905 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:10.898+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:11.020 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:11.013+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:11.095 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:11.088+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:11.134 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:11.128+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:11.183 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:11.176+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:11.322 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:11.315+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:11.915 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:11.909+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:11.939 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:11.934+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:12.049 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:12.040+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:12.115 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:12.107+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:12.116 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:12.108+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:12.181 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:12.176+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:12.370 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:12.364+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:12.920 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:12.912+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:12.934 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:12.928+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:13.081 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:13.075+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:13.133 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:13.127+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:13.157 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:13.149+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:13.165 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:13.158+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:13.372 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:13.366+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:13.939 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:13.932+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:13.942 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:13.936+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:14.048 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:14.042+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:14.090 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:14.084+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:14.176 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:14.169+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:14.196 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:14.189+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:14.384 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:14.377+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:14.945 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:14.939+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:14.971 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:14.963+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:15.066 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:15.059+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:15.075 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:15.068+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:15.154 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:15.148+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:15.194 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:15.187+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:15.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:15.390+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:15.994 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:15.987+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:15.998 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:15.992+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:16.055 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:16.050+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:16.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:16.070+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:16.126 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:16.121+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:16.241 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:16.220+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:16.365 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:16.358+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:16.986 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:16.980+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:17.021 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:17.014+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:17.036 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:17.029+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:17.079 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:17.073+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:17.103 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:17.097+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:17.218 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:17.208+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:17.319 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:17.313+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:17.984 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:17.974+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:18.006 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:18.001+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:18.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:18.073+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:18.113 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:18.104+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:18.139 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:18.133+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:18.249 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:18.239+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:18.348 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:18.341+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:18.992 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:18.985+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:19.035 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:19.026+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:19.077 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:19.068+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:19.088 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:19.079+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:19.108 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:19.103+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:19.269 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:19.262+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:19.378 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:19.369+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:20.029 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:20.023+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:20.029 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:20.023+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:20.034 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:20.029+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:20.086 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:20.078+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:20.104 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:20.097+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:20.322 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:20.315+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:20.351 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:20.345+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:21.049 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:21.043+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:21.049 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:21.042+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:21.078 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:21.072+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:21.086 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:21.081+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:21.097 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:21.092+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:21.368 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:21.362+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:21.375 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:21.368+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:22.050 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:22.044+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:22.067 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:22.059+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:22.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:22.087+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:22.100 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:22.093+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:22.100 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:22.094+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:22.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:22.368+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:22.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:22.393+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:23.024 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:23.018+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:23.059 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:23.053+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:23.119 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:23.113+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:23.120 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:23.115+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:23.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:23.130+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:23.413 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:23.406+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:23.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:23.444+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:24.013 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:24.004+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:24.092 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:24.086+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:24.093 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:24.087+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:24.102 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:24.098+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:24.103 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:24.098+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:24.429 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:24.421+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:24.490 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:24.483+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:25.035 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:25.030+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:25.064 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:25.055+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:25.065 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:25.059+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:25.079 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:25.073+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:25.131 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:25.125+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:25.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:25.389+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:25.484 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:25.478+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:26.045 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:26.036+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:26.056 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:26.048+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:26.058 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:26.050+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:26.091 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:26.085+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:26.143 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:26.137+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:26.408 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:26.401+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:26.483 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:26.476+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:27.054 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:27.048+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:27.090 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:27.085+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:27.093 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:27.086+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:27.096 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:27.090+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:27.176 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:27.170+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:27.401 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:27.395+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:27.461 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:27.456+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:28.076 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:28.071+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:28.090 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:28.085+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:28.108 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:28.103+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:28.136 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:28.131+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:28.230 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:28.225+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:28.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:28.399+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:28.467 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:28.454+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:29.074 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:29.069+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:29.087 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:29.082+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:29.140 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:29.134+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:29.152 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:29.147+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:29.248 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:29.242+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:29.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:29.371+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:29.450 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:29.444+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:30.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:30.049+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:30.117 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:30.101+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:30.129 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:30.123+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:30.186 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:30.178+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:30.300 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:30.295+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:30.388 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:30.379+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:30.476 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:30.472+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:31.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:31.002+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:31.148 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:31.138+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:31.157 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:31.149+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:31.235 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:31.228+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:31.317 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:31.311+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:31.437 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:31.427+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:31.437 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:31.430+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:32.015 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:32.009+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:32.111 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:32.106+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:32.134 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:32.129+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:32.217 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:32.211+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:32.277 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:32.273+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:32.435 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:32.429+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:32.456 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:32.442+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:33.044 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:33.038+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:33.145 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:33.139+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:33.159 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:33.154+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:33.189 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:33.181+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:33.275 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:33.269+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:33.396 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:33.387+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:33.464 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:33.459+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:34.069 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:34.063+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:34.179 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:34.173+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:34.196 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:34.191+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:34.199 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:34.194+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:34.242 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:34.232+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:34.372 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:34.366+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:34.486 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:34.479+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:35.089 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:35.080+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:35.172 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:35.157+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:35.218 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:35.212+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:35.239 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:35.235+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:35.249 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:35.244+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:35.397 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:35.387+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:35.500 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:35.496+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:36.068 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:36.064+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:36.140 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:36.135+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:36.209 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:36.204+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:36.223 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:36.218+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:36.295 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:36.289+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:36.432 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:36.427+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:36.517 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:36.513+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:37.098 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:37.092+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:37.111 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:37.105+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:37.171 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:37.164+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:37.270 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:37.263+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:37.297 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:37.291+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:37.428 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:37.423+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:37.495 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:37.487+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:38.105 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:38.098+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:38.126 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:38.119+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:38.208 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:38.201+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:38.272 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:38.266+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:38.336 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:38.326+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:38.398 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:38.391+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:38.518 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:38.511+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:39.102 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:39.094+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:39.120 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:39.112+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:39.208 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:39.202+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:39.313 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:39.307+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:39.364 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:39.358+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:39.436 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:39.431+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:39.484 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:39.476+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:40.059 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:40.054+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:40.117 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:40.104+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:40.261 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:40.248+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:40.355 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:40.349+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:40.417 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:40.404+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:40.468 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:40.459+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:40.502 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:40.497+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:41.076 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:41.070+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:41.084 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:41.079+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:41.302 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:41.297+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:41.317 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:41.308+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:41.400 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:41.394+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:41.461 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:41.456+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:41.514 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:41.509+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:42.077 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:42.071+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:42.117 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:42.111+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:42.332 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:42.326+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:42.336 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:42.331+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:42.357 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:42.351+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:42.473 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:42.467+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:42.494 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:42.488+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:43.009 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:41:43.005+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e294 v294) 2019-12-09T18:41:43.110 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:43.104+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:43.118 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:43.113+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:43.302 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:43.295+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:43.330 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:43.324+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:43.357 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:43.352+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:43.497 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:43.485+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:43.500 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:43.492+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:44.111 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:44.103+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:44.162 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:44.157+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:44.291 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:44.285+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:44.294 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:44.288+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:44.354 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:44.348+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:44.478 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:44.472+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:44.497 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:44.492+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:45.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:45.098+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:45.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:45.147+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:45.270 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:45.265+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:45.306 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:45.301+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:45.335 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:45.329+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:45.455 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:45.450+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:45.517 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:45.511+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:46.082 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:46.075+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:46.156 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:46.148+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:46.266 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:46.260+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:46.307 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:46.300+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:46.326 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:46.318+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:46.435 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:46.428+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:46.549 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:46.538+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:47.038 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:47.033+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:47.124 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:47.116+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:47.257 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:41:47.250+0000 1694f700 -1 osd.6 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.816243+0000 front 2019-12-09T18:40:28.067436+0000 (oldest deadline 2019-12-09T18:41:07.994467+0000) 2019-12-09T18:41:47.277 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:47.271+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:47.293 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:41:47.287+0000 1694f700 -1 osd.5 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.552986+0000 front 2019-12-09T18:40:25.787656+0000 (oldest deadline 2019-12-09T18:41:10.786334+0000) 2019-12-09T18:41:47.434 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:41:47.429+0000 1694f700 -1 osd.1 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:27.558300+0000 front 2019-12-09T18:40:26.120154+0000 (oldest deadline 2019-12-09T18:41:09.431338+0000) 2019-12-09T18:41:47.510 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:47.505+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:41:48.090 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:41:48.086+0000 1694f700 -1 osd.2 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:22.150109+0000 front 2019-12-09T18:40:27.524215+0000 (oldest deadline 2019-12-09T18:41:07.427972+0000) 2019-12-09T18:41:48.098 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:41:48.092+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 45sec e294 v294) 2019-12-09T18:41:48.196 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:41:48.185+0000 1694f700 -1 osd.4 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:25.799135+0000 front 2019-12-09T18:40:27.528675+0000 (oldest deadline 2019-12-09T18:41:09.271619+0000) 2019-12-09T18:41:48.239 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:41:48.231+0000 1694f700 -1 osd.7 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:26.564353+0000 front 2019-12-09T18:40:26.907632+0000 (oldest deadline 2019-12-09T18:41:09.368350+0000) 2019-12-09T18:41:48.555 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:41:48.549+0000 1694f700 -1 osd.0 294 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:40:23.969126+0000 front 2019-12-09T18:40:24.080147+0000 (oldest deadline 2019-12-09T18:41:09.230984+0000) 2019-12-09T18:42:31.330 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:31.235+0000 3827c700 -1 osd.1 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.372 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:31.363+0000 33a73700 -1 osd.7 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.398 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:31.386+0000 11caf700 -1 osd.7 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.423 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:31.326+0000 38a7d700 -1 osd.1 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.537 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:31.418+0000 35276700 -1 osd.1 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.592 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:31.470+0000 3827c700 -1 osd.7 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.636 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:31.427+0000 39a7f700 -1 osd.1 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.665 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:31.642+0000 35276700 -1 osd.7 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.942 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:31.700+0000 3aa81700 -1 osd.1 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:31.994 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:31.840+0000 3aa81700 -1 osd.1 298 build_incremental_map_msg missing incremental map 298 2019-12-09T18:42:32.728 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:32.718+0000 36278700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.084+0000 33a73700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.199+0000 34274700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.360 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.227+0000 3927e700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.364 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.238+0000 35276700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.423 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.283+0000 36278700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.586 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.391+0000 33a73700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.889 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.577+0000 36a79700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:33.894 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:33.582+0000 38a7d700 -1 osd.3 299 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:34.765 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:34.755+0000 35a77700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:34.982 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:34.970+0000 37a7b700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:35.327 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:35.120+0000 33a73700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:35.434 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:35.240+0000 3927e700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:35.514 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:35.321+0000 3aa81700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:35.825 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:35.575+0000 34274700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:36.429 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:36.412+0000 3aa81700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:38.110 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:37.902+0000 3727a700 -1 osd.3 300 build_incremental_map_msg missing incremental map 299 2019-12-09T18:42:38.151 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:42:37.926+0000 3727a700 -1 osd.3 300 build_incremental_map_msg missing incremental map 300 2019-12-09T18:42:44.476 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:44.471+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:45.443 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:45.437+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:45.742 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:45.737+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:46.401 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:46.395+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:46.721 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:46.715+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:47.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:47.391+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:47.408 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:47.403+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:47.680 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:47.674+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:47.699 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:59:44.318 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.712 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:59:44.331 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.732 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:59:44.351 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.733 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:59:44.332 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.739 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:59:44.338 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.745 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:59:44.364 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.959 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:08:59:44.557 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:47.967 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:08:59:44.585 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:48.020 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:08:59:44.638 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:42:48.280 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:48.276+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:48.355 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:48.351+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:48.449 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:48.445+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:48.695 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:48.690+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:49.275 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:49.271+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:49.389 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:49.385+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:49.483 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:49.478+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:49.716 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:49.712+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:50.229 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:50.223+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:50.309 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:50.304+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:50.440 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:50.435+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:50.477 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:50.471+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:50.735 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:50.730+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:51.269 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:51.264+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:51.331 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:51.326+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:51.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:51.408+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:51.491 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:51.487+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:51.714 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:51.709+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:52.313 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:52.308+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:52.321 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:52.316+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:52.386 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:52.380+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:52.497 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:52.492+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:52.720 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:52.715+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:53.289 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:53.283+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:53.310 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:53.305+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:53.357 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:53.351+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:53.518 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:53.514+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:53.710 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:53.706+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:54.341 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:54.334+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:54.342 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:54.337+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:54.399 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:54.393+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:54.515 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:54.510+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:54.689 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:54.683+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:55.306 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:55.302+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:55.393 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:55.388+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:55.441 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:55.436+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:55.498 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:55.492+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:55.680 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:55.675+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:56.310 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:56.305+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:56.366 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:56.361+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:56.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:56.412+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:56.452 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:56.447+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:56.691 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:56.685+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:57.361 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:57.355+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:57.391 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:57.387+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:57.436 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:57.431+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:57.473 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:57.467+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:57.674 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:57.670+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:58.329 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:58.323+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:58.442 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:58.438+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:58.449 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:58.443+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:58.469 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:58.462+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:58.686 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:58.681+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:42:59.361 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:42:59.356+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:42:59.439 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:42:59.433+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:42:59.457 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:42:59.452+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:42:59.474 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:42:59.468+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:42:59.660 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:42:59.656+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:00.361 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:00.355+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:00.458 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:00.450+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:00.502 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:00.497+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:00.505 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:00.501+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:00.672 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:00.668+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:01.375 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:01.371+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:01.430 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:01.425+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:01.540 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:01.536+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:01.556 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:01.550+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:01.744 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:01.738+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:02.429 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:02.422+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:02.469 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:02.464+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:02.516 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:02.511+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:02.534 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:02.529+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:02.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:02.732+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:03.141 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:03.136+0000 1694f700 -1 osd.3 300 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:43:03.451 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:03.445+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:03.458 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:03.453+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:03.560 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:43:03.270+0000 20e25700 -1 RGWWatcher::handle_error cookie 5396556848 err (107) Transport endpoint is not connected 2019-12-09T18:43:03.563 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:03.557+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:03.576 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:03.572+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:03.780 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:03.776+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:04.144 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:04.139+0000 1694f700 -1 osd.3 300 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:43:04.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:04.412+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:04.449 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:04.445+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:04.547 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:04.543+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:04.567 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:04.563+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:04.778 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:04.774+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:05.108 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:05.102+0000 1694f700 -1 osd.3 300 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:43:05.431 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:05.426+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:05.449 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:05.444+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:05.582 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:05.578+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:05.597 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:05.593+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:05.798 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:05.790+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:06.089 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:06.083+0000 1694f700 -1 osd.3 300 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:43:06.443 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:06.437+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:06.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:43:06.477+0000 1694f700 -1 osd.4 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123612 5.3 5:c609908c:::notify.5:head [watch ping cookie 5396556848 gen 5] snapc 0=[] ondisk+write+known_if_redirected e299) 2019-12-09T18:43:06.495 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:06.490+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:06.623 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:06.618+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:06.642 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:06.637+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:06.752 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:06.747+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:07.111 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:07.105+0000 1694f700 -1 osd.3 300 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:43:07.405 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:07.400+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:07.513 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:43:07.509+0000 1694f700 -1 osd.4 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123612 5.3 5:c609908c:::notify.5:head [watch ping cookie 5396556848 gen 5] snapc 0=[] ondisk+write+known_if_redirected e299) 2019-12-09T18:43:07.516 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:07.511+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:07.657 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:07.652+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:07.667 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:43:07.663+0000 1694f700 -1 osd.2 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.613147+0000 front 2019-12-09T18:42:05.547076+0000 (oldest deadline 2019-12-09T18:42:47.137235+0000) 2019-12-09T18:43:07.713 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:07.708+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:08.065 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:08.060+0000 1694f700 -1 osd.3 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:42:27.815033+0000 (oldest deadline 2019-12-09T18:43:07.815033+0000) 2019-12-09T18:43:08.067 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:08.062+0000 1694f700 -1 osd.3 300 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:43:08.379 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:43:08.374+0000 1694f700 -1 osd.0 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.225819+0000 front 2019-12-09T18:42:03.489110+0000 (oldest deadline 2019-12-09T18:42:44.406781+0000) 2019-12-09T18:43:08.492 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:43:08.486+0000 1694f700 -1 osd.1 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.738354+0000 front 2019-12-09T18:42:04.736798+0000 (oldest deadline 2019-12-09T18:42:47.765752+0000) 2019-12-09T18:43:08.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:43:08.511+0000 1694f700 -1 osd.4 300 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123612 5.3 5:c609908c:::notify.5:head [watch ping cookie 5396556848 gen 5] snapc 0=[] ondisk+write+known_if_redirected e299) 2019-12-09T18:43:08.651 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:43:08.646+0000 1694f700 -1 osd.5 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:04.741607+0000 front 2019-12-09T18:42:05.641421+0000 (oldest deadline 2019-12-09T18:42:49.577800+0000) 2019-12-09T18:43:08.725 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:43:08.712+0000 1694f700 -1 osd.7 300 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:42:05.539262+0000 front 2019-12-09T18:42:03.576456+0000 (oldest deadline 2019-12-09T18:42:45.299923+0000) 2019-12-09T18:43:09.556 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:43:09.543+0000 1694f700 -1 osd.4 301 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123612 5.3 5:c609908c:::notify.5:head [watch ping cookie 5396556848 gen 5] snapc 0=[] ondisk+write+known_if_redirected e299) 2019-12-09T18:43:09.616 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:43:09.447+0000 1694f700 -1 osd.3 301 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:123386 2.27s0 2.d6809c27 (undecoded) ondisk+retry+write+known_if_redirected e298) 2019-12-09T18:45:10.306 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:10.302+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:11.277 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:11.270+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:12.256 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:12.253+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:13.259 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:13.254+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:14.241 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:14.237+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:15.228 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:15.224+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:16.231 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:16.226+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:17.264 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:17.260+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:18.231 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:18.227+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:19.247 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:19.243+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:20.242 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:20.236+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:21.233 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:21.228+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:22.226 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:22.221+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:23.241 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:23.219+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:24.243 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:24.239+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:25.203 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:25.198+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:26.193 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:26.188+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:27.175 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:27.170+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:28.152 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:28.142+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:29.137 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:29.133+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:30.185 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:30.180+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:30.978 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:30.971+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:31.170 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:31.162+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:31.590 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:31.583+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:31.978 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:31.970+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:32.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:32.152+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:32.327 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:32.320+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:32.607 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:32.602+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:32.988 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:32.981+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:33.168 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:33.161+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:33.169 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:33.164+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:33.265 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:33.256+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:33.367 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:33.362+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:33.603 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:33.595+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:33.974 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:33.968+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:34.219 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:34.214+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:34.220 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:34.215+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:34.277 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:34.272+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:34.403 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:34.399+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:34.563 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:34.558+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:34.897 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:34.891+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:34.943 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:34.936+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:35.210 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:35.203+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:35.212 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:35.207+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:35.269 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:35.262+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:35.450 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:35.446+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:35.589 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:35.583+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:35.939 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:35.932+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:35.946 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:35.939+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:36.249 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:36.241+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:36.251 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:36.244+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:36.256 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:36.249+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:36.413 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:36.408+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:36.640 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:36.635+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:36.673 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:36.667+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:36.958 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:36.935+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:36.975 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:36.959+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:37.239 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:37.233+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:37.263 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:37.258+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:37.267 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:37.262+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:37.465 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:37.458+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:37.650 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:37.645+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:37.662 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:37.655+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:37.946 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:37.940+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:37.979 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:37.974+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:38.227 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:38.220+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:38.227 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:38.222+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:38.263 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:38.257+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:38.503 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:38.498+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:38.683 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:38.678+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:38.692 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:38.684+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:38.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:38.934+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:38.989 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:38.980+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:39.242 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:39.237+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:39.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:39.253+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:39.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:39.255+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:39.497 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:39.491+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:39.653 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:39.647+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:39.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:39.725+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:39.911 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:39.903+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:39.950 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:39.943+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:40.255 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:40.249+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:40.257 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:40.253+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:40.284 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:40.278+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:40.481 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:40.474+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:40.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:40.682+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:40.724 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:40.718+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:40.895 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:40.890+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:40.943 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:40.937+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:41.269 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:41.263+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:41.278 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:41.272+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:41.307 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:41.301+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:41.474 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:41.467+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:41.693 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:41.683+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:41.721 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:41.712+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:41.948 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:41.941+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:41.989 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:41.983+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:42.275 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:42.269+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:42.278 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:42.274+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:42.317 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:42.311+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:42.518 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:42.511+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:42.710 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:42.704+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:42.773 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:42.760+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:42.916 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:42.909+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:42.978 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:42.971+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:43.247 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:43.236+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:43.247 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:43.241+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:43.336 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:43.330+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:43.502 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:43.497+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:43.667 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:43.661+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:43.761 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:43.750+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:43.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:43.931+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:44.008 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:44.002+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:44.216 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:44.209+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:44.218 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:44.212+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:44.388 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:44.380+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:44.495 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:44.487+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:44.624 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:44.618+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:44.763 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:44.757+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:44.909 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:44.904+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:45.008 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:45.002+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:45.199 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:45.192+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:45.200 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:45.195+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:45.432 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:45.426+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:45.501 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:45.495+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:45.624 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:45.617+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:45.794 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:45.788+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:45.944 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:45.935+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:46.033 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:46.027+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:46.246 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:46.240+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:46.249 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:46.244+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:46.454 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:46.448+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:46.495 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:46.489+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:46.599 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:46.594+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:46.778 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:46.772+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:46.957 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:46.950+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:47.016 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:47.009+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:47.280 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:47.274+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:47.283 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:47.279+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:47.414 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:47.409+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:47.491 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:47.486+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:47.611 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:47.606+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:47.739 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:47.734+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:47.943 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:47.937+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:48.068 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:48.061+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:48.306 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:48.301+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:48.309 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:48.305+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:48.401 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:48.395+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:48.537 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:48.530+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:48.616 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:48.610+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:48.792 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:48.785+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:48.954 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:48.948+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:49.118 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:49.112+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:49.346 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:49.341+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:49.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:49.344+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:49.377 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:49.371+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:49.530 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:49.524+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:49.643 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:49.636+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:49.799 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:49.791+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:49.983 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:49.977+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:50.172 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:50.166+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:50.378 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:50.373+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:50.382 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:50.377+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:50.411 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:50.404+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:50.575 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:50.569+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:50.682 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:50.676+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:50.774 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:50.767+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:51.016 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:51.005+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:51.187 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:51.181+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:51.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:51.343+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:51.352 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:51.348+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:51.381 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:51.376+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:51.577 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:51.570+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:51.673 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:51.667+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:51.824 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:51.819+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:51.985 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:51.978+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:52.179 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:52.172+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:52.365 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:52.358+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:52.405 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:52.399+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:52.408 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:52.403+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:52.629 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:52.621+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:52.708 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:52.702+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:52.850 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:52.842+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:52.994 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:52.987+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:53.199 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:53.193+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:53.413 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:53.405+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:53.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:53.410+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:53.418 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:53.413+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:53.630 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:53.625+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:53.690 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:53.684+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:53.836 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:53.830+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:54.014 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:54.008+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:54.241 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:54.234+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:54.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:54.378+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:54.386 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:54.382+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:54.392 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:54.386+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:54.601 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:54.595+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:54.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:54.680+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:54.884 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:54.879+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:55.025 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:55.018+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:55.249 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:55.241+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:55.404 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:55.398+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:55.405 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:55.400+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:55.416 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:55.406+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:55.587 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:55.580+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:55.654 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:55.647+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:55.844 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:55.837+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:55.984 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:55.977+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:56.222 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:56.215+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:56.388 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:56.381+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:56.388 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:56.383+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:56.437 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:56.431+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:56.580 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:56.571+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:56.629 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:56.621+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:56.847 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:56.841+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:56.962 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:56.956+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:57.238 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:57.233+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:57.363 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:57.356+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:57.366 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:57.360+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:57.427 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:57.421+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:57.545 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:57.537+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:57.616 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:57.609+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:57.861 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:57.856+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:57.940 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:57.932+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:58.213 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:58.208+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:58.383 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:58.374+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:58.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:58.376+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:58.453 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:58.447+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:58.562 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:58.555+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:58.654 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:58.648+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:58.921 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:58.911+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:58.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:58.935+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:45:59.202 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:45:59.196+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:45:59.385 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:59.379+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:45:59.385 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:45:59.381+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:45:59.437 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:45:59.432+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:45:59.577 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:45:59.571+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:45:59.683 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:45:59.675+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:45:59.891 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:45:59.887+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:45:59.913 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:45:59.894+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:00.231 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:00.225+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:00.415 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:00.409+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:00.416 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:00.411+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:00.447 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:00.443+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:00.549 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:00.543+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:00.728 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:00.721+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:00.912 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:00.906+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:00.930 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:00.925+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:01.203 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:01.198+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:01.412 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:01.406+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:01.434 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:01.429+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:01.435 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:01.431+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:01.602 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:01.595+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:01.697 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:01.691+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:01.874 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:01.867+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:01.931 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:01.922+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:02.228 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:02.223+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:02.431 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:02.426+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:02.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:02.456+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:02.463 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:02.459+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:02.635 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:02.628+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:02.677 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:02.670+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:02.869 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:02.862+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:02.901 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:02.895+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:03.249 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:03.245+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:03.449 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:03.443+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:03.498 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:03.493+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:03.499 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:03.495+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:03.661 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:03.655+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:03.670 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:03.664+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:03.859 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:03.852+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:03.905 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:03.900+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:04.213 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:46:04.204+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e306 v306) 2019-12-09T18:46:04.279 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:04.275+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:04.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:04.420+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:04.499 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:04.495+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:04.502 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:04.499+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:04.663 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:04.656+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:04.676 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:04.670+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:04.863 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:04.857+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:04.941 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:04.935+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:05.240 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:05.235+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:05.400 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:05.394+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:05.537 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:05.533+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:05.540 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:05.537+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:05.636 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:05.626+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:05.657 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:05.643+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:05.875 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:05.869+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:05.931 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:05.927+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:06.212 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:06.206+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:06.418 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:06.412+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:06.558 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:06.551+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:06.561 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:06.557+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:06.676 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:06.669+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:06.683 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:06.677+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:06.872 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:06.865+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:06.979 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:06.973+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:07.194 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:07.187+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:07.428 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:07.421+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:07.576 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:07.568+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:07.577 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:07.571+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:07.649 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:07.640+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:07.703 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:07.696+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:07.901 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:07.894+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:07.987 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:07.981+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:08.224 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:08.217+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:08.387 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:08.381+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:08.569 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:08.562+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:08.570 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:08.565+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:08.686 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:08.680+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:08.690 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:08.683+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:08.862 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:08.855+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:08.964 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:08.957+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:09.181 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:09.175+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:09.231 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:46:09.223+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e306 v306) 2019-12-09T18:46:09.411 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:09.406+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:09.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:09.594+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:09.601 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:09.596+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:09.686 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:09.679+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:09.695 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:09.688+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:09.888 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:09.881+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:09.940 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:09.934+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:10.146 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:10.140+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:10.405 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:10.399+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:10.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:10.627+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:10.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:10.630+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:10.657 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:10.650+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:10.677 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:10.664+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:10.885 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:10.878+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:10.985 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:10.979+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:11.115 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:11.109+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:11.361 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:11.355+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:11.623 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:11.616+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:11.624 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:11.619+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:11.671 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:11.665+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:11.678 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:11.671+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:11.897 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:11.891+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:12.023 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:12.017+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:12.145 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:12.133+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:12.355 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:12.349+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:12.627 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:12.619+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:12.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:12.628+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:12.638 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:12.632+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:12.638 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:12.633+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:12.872 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:12.865+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:13.003 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:12.997+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:13.174 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:13.168+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:13.358 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:13.352+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:13.614 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:13.607+0000 1694f700 -1 osd.4 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:50.380353+0000 front 2019-12-09T18:44:50.437263+0000 (oldest deadline 2019-12-09T18:45:32.736439+0000) 2019-12-09T18:46:13.614 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:13.609+0000 1694f700 -1 osd.4 306 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:13.638 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:46:13.631+0000 1694f700 -1 osd.2 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.533205+0000 front 2019-12-09T18:44:47.591806+0000 (oldest deadline 2019-12-09T18:45:32.012841+0000) 2019-12-09T18:46:13.642 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:13.634+0000 1694f700 -1 osd.1 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.151759+0000 front 2019-12-09T18:44:48.433434+0000 (oldest deadline 2019-12-09T18:45:31.194495+0000) 2019-12-09T18:46:13.915 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:46:13.907+0000 1694f700 -1 osd.0 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.150053+0000 front 2019-12-09T18:44:48.794020+0000 (oldest deadline 2019-12-09T18:45:34.401764+0000) 2019-12-09T18:46:14.031 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:46:14.024+0000 1694f700 -1 osd.7 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.829000+0000 front 2019-12-09T18:44:51.140291+0000 (oldest deadline 2019-12-09T18:45:35.700771+0000) 2019-12-09T18:46:14.225 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:46:14.219+0000 1694f700 -1 osd.6 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:49.467066+0000 front 2019-12-09T18:44:52.008441+0000 (oldest deadline 2019-12-09T18:45:30.953561+0000) 2019-12-09T18:46:14.303 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:46:14.296+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e306 v306) 2019-12-09T18:46:14.331 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:46:14.325+0000 1694f700 -1 osd.5 306 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:44:51.142428+0000 front 2019-12-09T18:44:51.150001+0000 (oldest deadline 2019-12-09T18:45:32.967567+0000) 2019-12-09T18:46:14.829 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:14.778+0000 1694f700 -1 osd.4 307 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e306) 2019-12-09T18:46:16.226 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:46:16.221+0000 1694f700 -1 osd.4 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:123839 2.1fs0 2:f939b221:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.631_alpha%2f306119d5a5e9465f90686d883d856f0b:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e306) 2019-12-09T18:46:49.042 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:46:48.990+0000 2825c700 -1 reset not still connected to 0x3e1b2970 2019-12-09T18:46:50.596 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:03:47.213 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.605 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:03:47.224 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.623 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:03:47.241 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.624 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:03:47.223 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.627 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:03:47.245 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.629 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:03:47.228 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.887 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:03:47.505 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.894 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:03:47.492 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:50.895 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:03:47.514 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:46:53.480 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.422+0000 34a75700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:46:53.778 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.594+0000 35276700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:46:53.905 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.617+0000 39a7f700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:46:53.927 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.707+0000 34274700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:46:53.936 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.754+0000 3aa81700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:46:53.976 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.808+0000 34a75700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:46:54.011 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:46:53.864+0000 3aa81700 -1 osd.1 310 build_incremental_map_msg missing incremental map 310 2019-12-09T18:47:23.238 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:23.232+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:24.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:24.247+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:25.299 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:25.292+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:26.289 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:26.284+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:27.317 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:27.311+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:28.279 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:28.273+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:29.312 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:29.305+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:30.359 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:30.351+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:31.404 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:31.398+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:32.452 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:32.446+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:33.485 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:33.479+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:34.505 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:34.496+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:35.507 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:35.501+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:36.494 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:36.487+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:37.530 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:37.524+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:38.529 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:38.523+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:39.568 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:39.562+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:40.534 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:40.528+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:41.023 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:47:40.793+0000 20e25700 -1 RGWWatcher::handle_error cookie 1047330704 err (107) Transport endpoint is not connected 2019-12-09T18:47:41.552 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:41.546+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:42.581 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:42.576+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:43.617 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:43.611+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:44.661 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:44.649+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:45.278 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:45.273+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:45.633 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:45.627+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:46.274 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:46.269+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:46.616 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:46.610+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:47.227 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:47.223+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:47.670 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:47.664+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:48.199 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:48.194+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:48.629 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:48.623+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:49.242 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:49.237+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:49.683 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:49.678+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:50.258 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:50.253+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:50.659 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:50.652+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:51.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:51.268+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:51.627 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:51.615+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:52.251 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:52.246+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:52.618 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:52.611+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:53.244 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:53.238+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:53.624 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:53.618+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:54.199 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:54.195+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:54.667 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:54.662+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:55.172 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:55.167+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:55.682 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:55.677+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:55.758 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:47:55.750+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:47:56.190 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:56.185+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:56.681 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:47:56.673+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:47:56.705 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:56.694+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:56.763 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:47:56.756+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:47:57.163 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:57.158+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:57.683 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:57.677+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:57.710 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:47:57.705+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:47:57.744 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:47:57.738+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:47:57.977 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:47:57.966+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:47:58.139 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:58.132+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:47:58.140 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:58.135+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:58.699 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:47:58.694+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:47:58.711 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:58.707+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:58.745 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:47:58.739+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:47:58.943 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:47:58.939+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:47:59.162 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:59.156+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:47:59.163 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:47:59.157+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:47:59.747 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:47:59.742+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:47:59.756 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:59.751+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:47:59.759 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:47:59.755+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:47:59.795 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:47:59.790+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:47:59.995 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:47:59.989+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:00.206 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:00.200+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:00.208 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:00.204+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:00.408 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:00.403+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:00.752 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:00.746+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:00.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:00.764+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:00.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:00.766+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:00.802 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:00.797+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:01.020 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:01.014+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:01.232 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:01.226+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:01.233 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:01.227+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:01.375 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:01.369+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:01.791 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:01.786+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:01.794 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:01.789+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:01.799 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:01.794+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:01.827 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:01.821+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:02.023 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:02.018+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:02.241 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:02.234+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:02.243 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:02.236+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:02.404 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:02.398+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:02.490 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:02.485+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:02.795 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:02.786+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:02.795 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:02.788+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:02.841 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:02.833+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:02.870 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:02.865+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:03.055 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:03.049+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:03.207 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:03.201+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:03.207 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:03.202+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:03.390 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:03.377+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:03.518 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:03.513+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:03.813 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:03.807+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:03.817 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:03.812+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:03.864 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:03.859+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:03.918 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:03.913+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:04.069 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:04.062+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:04.189 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:04.179+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:04.190 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:04.181+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:04.405 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:04.400+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:04.504 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:04.499+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:04.781 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:04.776+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:04.784 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:04.779+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:04.874 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:04.870+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:04.896 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:05:01.515 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:04.902 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:05:01.520 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:04.935 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:05:01.534 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:04.949 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:05:01.567 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:04.961 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:04.954+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:04.965 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:05:01.562 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:04.972 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:05:01.588 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:05.032 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:05:01.650 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:05.060 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:05:01.679 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:05.066 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:05:01.664 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:05.117 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:05.112+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:05.232 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:05.224+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:05.232 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:05.226+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:05.429 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:05.422+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:05.543 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:05.538+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:05.815 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:05.809+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:05.815 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:05.810+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:05.868 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:05.863+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:05.933 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:05.926+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:06.093 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:06.087+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:06.240 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:06.234+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:06.240 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:06.236+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:06.457 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:06.451+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:06.549 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:06.541+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:06.793 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:06.788+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:06.795 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:06.791+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:06.900 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:06.895+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:06.946 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:06.941+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:07.144 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:07.137+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:07.229 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:07.221+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:07.229 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:07.223+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:07.463 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:07.457+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:07.504 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:07.498+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:07.762 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:07.755+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:07.762 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:07.757+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:07.895 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:07.883+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:07.918 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:07.910+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:08.105 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:08.099+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:08.269 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:08.264+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:08.271 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:08.266+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:08.439 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:08.433+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:08.510 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:08.505+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:08.770 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:08.763+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:08.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:08.766+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:08.951 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:08.936+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:08.969 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:08.963+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:09.095 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:09.090+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:09.283 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:09.276+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:09.304 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:09.289+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:09.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:09.420+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:09.527 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:09.522+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:09.742 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:09.735+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:09.743 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:09.737+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:09.903 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:09.898+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:09.986 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:09.979+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:10.133 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:10.128+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:10.338 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:10.332+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:10.339 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:10.334+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:10.453 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:10.447+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:10.546 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:10.541+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:10.757 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:10.750+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:10.759 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:10.752+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:10.891 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:10.886+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:11.033 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:11.028+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:11.128 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:11.122+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:11.322 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:11.316+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:11.326 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:11.321+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:11.415 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:11.409+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:11.598 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:11.592+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:11.755 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:11.750+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:11.758 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:11.754+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:11.920 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:11.916+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:11.995 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:11.991+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:12.089 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:12.082+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:12.343 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:12.339+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:12.345 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:12.341+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:12.459 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:12.454+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:12.579 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:12.572+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:12.744 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:12.737+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:12.745 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:12.740+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:12.973 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:12.968+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:12.977 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:12.973+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:13.139 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:13.132+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:13.396 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:13.391+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:13.397 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:13.393+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:13.414 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:13.410+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:13.559 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:13.552+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:13.783 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:13.777+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:13.783 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:13.778+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:13.949 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:13.943+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:13.982 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:13.978+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:14.179 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:14.174+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:14.373 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:14.365+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:14.373 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:14.367+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:14.443 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:14.438+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:14.526 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:14.520+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:14.823 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:14.818+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:14.823 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:14.819+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:14.916 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:14.911+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:14.986 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:14.967+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:15.220 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:15.215+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:15.339 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:15.333+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:15.339 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:15.336+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:15.408 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:15.404+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:15.534 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:15.529+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:15.837 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:15.831+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:15.838 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:15.833+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:15.916 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:15.910+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:16.026 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:16.022+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:16.232 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:16.227+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:16.365 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:16.361+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:16.369 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:16.365+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:16.452 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:16.447+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:16.577 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:16.572+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:16.850 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:16.845+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:16.852 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:16.848+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:16.931 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:16.926+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:17.053 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:17.048+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:17.273 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:17.265+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:17.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:17.410+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:17.417 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:17.414+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:17.498 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:17.493+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:17.541 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:17.534+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:17.841 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:17.836+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:17.842 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:17.837+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:17.908 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:17.903+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:18.073 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:18.068+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:18.315 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:18.307+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:18.434 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:18.428+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:18.435 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:18.430+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:18.497 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:18.493+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:18.557 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:18.551+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:18.851 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:18.845+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:18.852 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:18.847+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:18.941 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:18.935+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:19.110 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:19.105+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:19.320 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:19.314+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:19.440 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:19.435+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:19.441 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:19.437+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:19.466 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:19.458+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:19.604 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:19.595+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:19.896 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:19.891+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:19.897 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:19.893+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:19.978 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:19.972+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:20.160 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:20.157+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:20.366 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:20.360+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:20.444 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:20.439+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:20.452 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:20.443+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:20.457 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:20.452+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:20.645 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:20.634+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:20.868 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:20.864+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:20.870 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:20.866+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:20.943 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:20.938+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:21.186 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:21.181+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:21.341 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:21.334+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:21.470 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:21.465+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:21.498 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:21.492+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:21.498 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:21.494+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:21.629 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:21.623+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:21.859 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:21.854+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:21.860 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:21.855+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:21.985 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:21.973+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:22.187 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:22.182+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:22.343 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:22.337+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:22.435 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:22.430+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:22.544 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:22.539+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:22.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:22.543+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:22.673 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:22.668+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:22.874 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:22.869+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:22.877 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:22.872+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:22.981 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:22.973+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:23.236 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:23.229+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:23.384 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:23.377+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:23.467 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:23.454+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:23.603 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:23.589+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:23.620 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:23.591+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:23.718 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:23.712+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:23.920 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:23.913+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:23.920 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:23.916+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:23.957 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:23.952+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:24.253 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:24.248+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:24.419 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:24.413+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:24.448 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:24.444+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:24.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:24.593+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:24.602 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:24.596+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:24.720 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:24.714+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:24.931 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:24.923+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:24.932 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:24.924+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:24.999 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:24.988+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:25.258 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:25.252+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:25.465 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:25.460+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:25.483 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:25.477+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:25.588 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:25.582+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:25.589 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:25.584+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:25.705 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:25.693+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:25.936 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:25.930+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:25.937 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:25.932+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:25.987 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:25.981+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:26.246 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:26.241+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:26.443 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:26.438+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:26.504 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:26.498+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:26.642 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:26.636+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:26.645 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:26.640+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:26.697 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:26.687+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:26.952 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:26.945+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:26.975 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:26.966+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:26.975 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:26.968+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:27.284 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:27.279+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:27.465 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:27.459+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:27.486 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:27.480+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:27.643 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:27.637+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:27.647 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:27.642+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:27.692 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:27.686+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:27.913 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:27.907+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:27.981 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:27.975+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:27.984 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:27.979+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:28.254 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:28.248+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:28.463 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:28.458+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:28.540 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:28.533+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:28.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:28.629+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:28.637 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:28.633+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:28.710 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:28.704+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:28.892 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:28.884+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:28.957 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:28.948+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:28.957 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:28.951+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:29.238 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:29.232+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:29.432 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:29.425+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:29.545 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:29.534+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:29.609 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:29.599+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:29.609 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:29.601+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:29.738 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:29.732+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:29.890 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:29.884+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:29.948 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:48:29.938+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e312 v312) 2019-12-09T18:48:29.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:29.951+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:29.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:29.954+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:30.283 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:30.278+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:30.412 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:48:30.404+0000 1694f700 -1 osd.5 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.513167+0000 front 2019-12-09T18:47:17.523468+0000 (oldest deadline 2019-12-09T18:47:59.684621+0000) 2019-12-09T18:48:30.501 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:30.495+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:30.602 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:30.597+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:30.605 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:30.600+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:30.765 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:30.757+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:30.883 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:30.875+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:31.009 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:30.999+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:31.009 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:31.003+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:31.328 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:31.321+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:47:14.504348+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:31.490 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:31.484+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:31.603 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:31.596+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:31.604 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:31.598+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:31.735 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:31.727+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:31.871 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:48:31.865+0000 1694f700 -1 osd.1 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:27.239237+0000 front 2019-12-09T18:47:14.493042+0000 (oldest deadline 2019-12-09T18:47:54.834528+0000) 2019-12-09T18:48:31.994 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:31.988+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:31.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:31.989+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:32.367 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:48:32.360+0000 1694f700 -1 osd.7 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.569755+0000 front 2019-12-09T18:48:31.874325+0000 (oldest deadline 2019-12-09T18:47:56.178394+0000) 2019-12-09T18:48:32.538 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:32.531+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:32.643 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:32.638+0000 1694f700 -1 osd.4 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:32.645 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:32.641+0000 1694f700 -1 osd.4 312 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:32.748 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:32.742+0000 1694f700 -1 osd.0 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:32.987 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:32.981+0000 1694f700 -1 osd.3 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.193046+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:32.989 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:32.984+0000 1694f700 -1 osd.3 312 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:33.516 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:48:33.510+0000 1694f700 -1 osd.2 312 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:12.637178+0000 front 2019-12-09T18:47:12.821879+0000 (oldest deadline 2019-12-09T18:47:57.580351+0000) 2019-12-09T18:48:33.653 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:33.643+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:47:16.583900+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:33.656 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:33.645+0000 1694f700 -1 osd.4 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:33.739 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:33.726+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:34.032 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:34.024+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:48:33.855939+0000 front 2019-12-09T18:47:17.206963+0000 (oldest deadline 2019-12-09T18:47:59.315431+0000) 2019-12-09T18:48:34.036 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:34.025+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:34.617 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:34.611+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:16.592558+0000 front 2019-12-09T18:48:34.571311+0000 (oldest deadline 2019-12-09T18:47:57.668889+0000) 2019-12-09T18:48:34.617 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:34.612+0000 1694f700 -1 osd.4 313 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124108 5.3 5:c609908c:::notify.5:head [watch ping cookie 1047330704 gen 2] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:34.703 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:34.696+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:35.001 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:34.995+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:35.581 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:48:35.576+0000 1694f700 -1 osd.4 313 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124143 5.3 5:c609908c:::notify.5:head [watch unwatch cookie 1047330704] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-09T18:48:35.677 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:48:35.671+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:47:17.535611+0000 front 2019-12-09T18:47:17.548298+0000 (oldest deadline 2019-12-09T18:48:01.436534+0000) 2019-12-09T18:48:36.023 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:36.016+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:37.195 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:37.173+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:38.207 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:38.202+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:39.222 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:39.215+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:40.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:40.197+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:41.209 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:41.199+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:42.202 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:42.194+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:43.182 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:43.174+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:44.187 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:44.181+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:45.222 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:45.216+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:46.207 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:46.201+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:47.231 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:47.226+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:48.267 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:48.262+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:49.259 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:49.252+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:50.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:50.219+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:51.270 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:51.264+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:52.304 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:52.299+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:53.309 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:53.303+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:54.280 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:54.274+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:55.251 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:55.245+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:56.232 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:56.226+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:57.247 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:57.241+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:58.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:58.250+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:48:59.257 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:48:59.251+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:00.277 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:00.271+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:01.363 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:01.356+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:02.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:02.343+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:03.370 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:03.363+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:04.597 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:04.579+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:05.749 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:05.606+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:06.901 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:06.825+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:07.917 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:07.881+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:08.890 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:49:08.882+0000 1694f700 -1 osd.3 313 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=46 ondisk+retry+read+known_if_redirected e310) 2019-12-09T18:49:59.696 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:49:59.689+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:49:59.968 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:49:59.961+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:49:59.977 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:49:59.970+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:00.147 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:00.136+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:00.549 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:00.543+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:00.686 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:00.680+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:00.947 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:00.941+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:00.972 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:00.965+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:01.200 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:01.194+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:01.602 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:01.595+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:01.686 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:01.681+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:01.924 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:01.919+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:01.982 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:01.976+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:02.176 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:02.169+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:02.615 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:02.610+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:02.628 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:02.621+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:02.735 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:02.728+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:02.981 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:02.971+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:03.012 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:03.005+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:03.070 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:03.060+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:03.183 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:03.177+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:03.598 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:03.592+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:03.637 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:03.630+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:03.774 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:03.769+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:03.972 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:03.966+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:04.022 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:04.015+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:04.025 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:04.019+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:04.184 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:04.179+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:04.628 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:04.621+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:04.636 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:04.629+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:04.753 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:04.742+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:04.986 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:04.977+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:05.002 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:04.994+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:05.071 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:05.063+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:05.240 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:05.233+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:05.611 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:05.604+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:05.622 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:05.616+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:05.746 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:05.740+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:06.031 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:06.024+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:06.031 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:06.023+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:06.080 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:06.073+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:06.228 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:06.222+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:06.639 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:06.634+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:06.665 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:06.657+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:06.786 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:06.779+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:07.031 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:07.022+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:07.042 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:07.036+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:07.122 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:07.116+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:07.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:07.276+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:07.687 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:07.677+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:07.710 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:07.703+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:07.839 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:07.834+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:08.018 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:08.010+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:08.022 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:08.015+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:08.174 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:08.168+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:08.274 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:08.268+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:08.723 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:08.717+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:08.757 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:08.750+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:08.825 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:08.819+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:08.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:08.969+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:09.049 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:09.043+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:09.166 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:09.161+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:09.283 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:09.277+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:09.713 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:09.705+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:09.724 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:09.718+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:09.798 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:09.792+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:09.994 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:09.988+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:10.013 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:10.005+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:10.147 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:10.140+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:10.309 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:10.303+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:10.703 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:10.697+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:10.732 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:10.725+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:10.779 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:10.770+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:10.956 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:10.949+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:11.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:11.021+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:11.163 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:11.156+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:11.300 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:11.293+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:11.716 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:11.710+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:11.739 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:11.725+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:11.835 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:11.828+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:12.004 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:11.997+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:12.049 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:12.041+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:12.178 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:12.172+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:12.343 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:12.337+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:12.684 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:12.678+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:12.752 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:12.744+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:12.841 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:12.826+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:13.054 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:13.048+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:13.081 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:13.074+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:13.140 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:13.134+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:13.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:13.343+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:13.735 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:13.729+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:13.789 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:13.782+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:13.825 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:13.819+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:14.021 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:14.014+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:14.076 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:14.070+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:14.152 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:14.146+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:14.391 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:14.384+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:14.735 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:14.727+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:14.807 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:14.801+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:14.833 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:14.826+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:15.053 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:15.042+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:15.122 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:15.107+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:15.123 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:15.058+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:15.378 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:15.372+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:15.692 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:15.685+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:15.842 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:15.828+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:15.848 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:15.842+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:16.015 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:16.009+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:16.099 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:16.092+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:16.106 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:16.100+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:16.380 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:16.373+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:16.739 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:16.732+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:16.848 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:16.841+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:16.855 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:16.848+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:17.032 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:17.025+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:17.149 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:17.141+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:17.152 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:17.146+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:17.360 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:17.354+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:17.722 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:17.716+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:17.832 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:17.826+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:17.876 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:17.870+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:18.038 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:18.030+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:18.164 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:18.157+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:18.185 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:18.179+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:18.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:18.343+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:18.684 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:18.678+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:18.821 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:18.814+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:18.901 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:18.895+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:19.062 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:19.055+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:19.148 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:19.142+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:19.154 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:19.148+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:19.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:19.389+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:19.644 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:19.638+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:19.801 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:19.796+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:19.918 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:19.913+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:20.062 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:20.052+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:20.150 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:20.144+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:20.167 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:20.160+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:20.372 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:20.366+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:20.655 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:20.649+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:20.826 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:20.818+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:20.968 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:20.962+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:21.108 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:21.102+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:21.109 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:21.104+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:21.134 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:21.128+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:21.414 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:21.409+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:21.649 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:21.644+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:21.852 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:21.845+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:21.998 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:21.991+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:22.075 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:22.068+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:22.111 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:22.103+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:22.113 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:22.103+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:22.441 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:22.431+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:22.716 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:22.705+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:22.869 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:22.861+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:22.965 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:22.958+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:23.050 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:23.044+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:23.138 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:23.132+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:23.139 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:23.133+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:23.426 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:23.420+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:23.752 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:23.740+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:23.853 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:23.844+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:23.951 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:23.940+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:24.061 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:24.054+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:24.150 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:24.144+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:24.184 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:24.178+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:24.430 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:24.423+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:24.799 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:24.793+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:24.831 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:24.825+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:24.949 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:24.942+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:25.042 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:25.035+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:25.121 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:25.100+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:25.177 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:25.170+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:25.425 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:25.416+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:25.766 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:25.760+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:25.832 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:25.825+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:25.935 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:25.929+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:26.064 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:26.058+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:26.160 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:26.154+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:26.207 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:26.200+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:26.445 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:26.439+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:26.728 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:26.722+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:26.824 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:26.818+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:26.988 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:26.984+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:27.073 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:27.067+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:27.182 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:27.177+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:27.241 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:27.231+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:27.468 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:27.463+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:27.683 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:27.678+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:27.845 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:27.839+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:27.958 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:27.953+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:28.044 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:28.037+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:28.198 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:28.190+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:28.239 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:28.233+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:28.496 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:28.492+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:28.675 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:28.670+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:28.860 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:28.854+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:29.009 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:29.005+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:29.111 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:29.100+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:29.199 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:29.191+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:29.230 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:29.224+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:29.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:29.489+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:29.708 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:29.703+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:29.877 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:29.871+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:30.031 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:30.025+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:30.078 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:30.070+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:30.215 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:30.209+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:30.222 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:30.216+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:30.342 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:50:30.335+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e313 v313) 2019-12-09T18:50:30.473 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:30.468+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:30.703 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:30.697+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:30.883 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:30.877+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:31.006 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:31.002+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:31.072 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:31.065+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:31.213 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:31.205+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:31.260 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:31.242+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:31.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:31.476+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:31.752 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:31.747+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:31.882 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:31.876+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:31.965 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:31.959+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:32.108 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:32.101+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:32.231 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:32.223+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:32.288 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:32.282+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:32.453 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:32.448+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:32.760 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:32.755+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:32.845 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:32.838+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:32.923 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:32.918+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:33.063 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:33.056+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:33.247 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:33.238+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:33.268 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:33.262+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:33.413 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:33.408+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:33.785 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:33.780+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:33.867 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:33.861+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:33.921 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:33.915+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:34.038 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:34.031+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:34.230 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:34.222+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:34.240 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:34.235+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:34.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:34.368+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:34.797 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:34.791+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:34.914 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:34.908+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:34.933 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:34.928+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:35.058 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:35.050+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:35.265 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:35.259+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:35.272 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:35.264+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:35.363 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:50:35.357+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e313 v313) 2019-12-09T18:50:35.368 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:35.362+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:35.773 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:35.769+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:35.881 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:35.876+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:35.898 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:35.893+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:36.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:36.093+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:36.264 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:36.258+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:36.311 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:36.305+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:36.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:36.342+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:36.793 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:36.787+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:36.843 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:36.837+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:36.947 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:36.941+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:37.105 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:50:37.096+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:19.185911+0000 front 2019-12-09T18:49:19.538782+0000 (oldest deadline 2019-12-09T18:49:59.516113+0000) 2019-12-09T18:50:37.229 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:37.223+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:37.277 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:37.271+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:37.404 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:37.398+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:37.754 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:37.748+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:37.863 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:37.858+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:37.914 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:37.908+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:38.246 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:38.240+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:38.281 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:38.275+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:38.368 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:38.362+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:38.804 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:38.798+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:38.883 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:38.877+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:38.900 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:38.894+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:39.260 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:39.254+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:39.270 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:39.265+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:39.400 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:39.393+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:39.764 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:39.758+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:39.875 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:50:39.868+0000 1694f700 -1 osd.5 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:14.682573+0000 front 2019-12-09T18:49:19.335773+0000 (oldest deadline 2019-12-09T18:49:59.282353+0000) 2019-12-09T18:50:39.883 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:50:39.878+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.864159+0000 front 2019-12-09T18:49:16.912959+0000 (oldest deadline 2019-12-09T18:50:02.063956+0000) 2019-12-09T18:50:40.218 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:50:40.212+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.874466+0000 front 2019-12-09T18:49:19.188584+0000 (oldest deadline 2019-12-09T18:50:02.117900+0000) 2019-12-09T18:50:40.278 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:50:40.273+0000 1694f700 -1 osd.2 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.270960+0000 front 2019-12-09T18:49:18.547252+0000 (oldest deadline 2019-12-09T18:49:59.804414+0000) 2019-12-09T18:50:40.420 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:50:40.412+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:18.425923+0000 front 2019-12-09T18:49:18.861486+0000 (oldest deadline 2019-12-09T18:49:59.774437+0000) 2019-12-09T18:50:40.449 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:50:40.439+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e313 v313) 2019-12-09T18:50:40.773 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:50:40.769+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:49:16.138829+0000 front 2019-12-09T18:49:16.146892+0000 (oldest deadline 2019-12-09T18:49:59.580702+0000) 2019-12-09T18:50:56.685 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:07:53.303 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.699 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:07:53.317 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.699 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:07:53.318 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.703 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:07:53.322 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.710 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:07:53.308 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.721 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:07:53.316 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.890 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:07:53.508 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:56.895 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:07:53.514 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:57.002 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:07:53.601 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.637 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:20.421+0000 34274700 -1 osd.4 317 build_incremental_map_msg missing incremental map 317 2019-12-09T18:51:20.733 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:20.425+0000 34274700 -1 osd.4 317 build_incremental_map_msg missing incremental map 317 2019-12-09T18:51:20.781 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:20.500+0000 33a73700 -1 osd.4 317 build_incremental_map_msg missing incremental map 317 2019-12-09T18:51:20.781 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:20.527+0000 35a77700 -1 osd.4 317 build_incremental_map_msg missing incremental map 317 2019-12-09T18:51:20.789 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:20.561+0000 34274700 -1 osd.4 317 build_incremental_map_msg missing incremental map 317 2019-12-09T18:51:20.847 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:08:17.466 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.878 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:08:17.477 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.881 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:08:17.500 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.908 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:08:17.527 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.916 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:08:17.514 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.927 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:08:17.539 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.960 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:08:17.579 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.969 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:08:17.567 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:20.987 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:08:17.606 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:21.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:21.049+0000 3aa81700 -1 osd.4 317 build_incremental_map_msg missing incremental map 317 2019-12-09T18:51:23.450 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.372+0000 38a7d700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.703 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.677+0000 3827c700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.709 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.698+0000 124b0700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.723 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.699+0000 36a79700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.771 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.717+0000 35276700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.772 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.721+0000 3927e700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.790 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.766+0000 35a77700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.832 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.780+0000 39a7f700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:23.841 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:23.821+0000 36a79700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:27.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:27.299+0000 3aa81700 -1 osd.3 319 build_incremental_map_msg missing incremental map 319 2019-12-09T18:51:33.994 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:51:33.989+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:51:33.198070+0000) 2019-12-09T18:51:34.981 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:51:34.970+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:51:33.198070+0000) 2019-12-09T18:51:35.961 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:51:35.957+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:51:33.198070+0000) 2019-12-09T18:51:36.969 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:51:36.963+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:51:33.198070+0000) 2019-12-09T18:51:37.092 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:37.085+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:38.057 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:38.050+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:38.118 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:38.112+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:38.862 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:38.854+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:39.083 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:39.076+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:39.155 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:39.149+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:39.839 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:39.834+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:40.058 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:40.052+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:40.145 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:40.137+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:40.851 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:40.846+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:41.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:41.089+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:41.162 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:41.156+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:41.294 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:41.289+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:41.906 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:41.900+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:42.125 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:42.118+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:42.140 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:42.133+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:42.204 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:42.198+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:42.311 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:42.305+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:42.962 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:42.957+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:43.105 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:43.093+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:43.149 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:43.139+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:43.222 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:43.216+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:43.333 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:43.321+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:43.939 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:43.932+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:44.101 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:44.095+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:44.111 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:44.105+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:44.257 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:44.252+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:44.310 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:44.305+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:44.936 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:44.931+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:45.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:45.051+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:45.098 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:45.091+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:45.251 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:45.245+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:45.317 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:45.312+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:45.963 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:45.957+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:46.114 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:46.107+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:46.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:46.126+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:46.253 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:46.241+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:46.359 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:46.354+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:46.985 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:46.979+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:47.129 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:47.123+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:47.143 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:47.138+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:47.310 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:47.303+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:47.344 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:47.339+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:47.993 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:47.983+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:48.158 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:48.152+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:48.176 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:48.169+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:48.333 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:48.326+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:48.339 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:48.323+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:48.987 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:48.982+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:49.153 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:49.147+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:49.180 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:49.173+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:49.348 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:49.342+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:49.355 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:49.349+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:50.036 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:50.030+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:50.119 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:50.112+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:50.164 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:50.157+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:50.359 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:50.353+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:50.368 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:50.361+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:50.614 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:50.606+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:51.057 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:51.052+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:51.142 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:51.137+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:51.207 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:51.201+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:51.402 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:51.396+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:51.405 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:51.398+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:51.579 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:51.573+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:52.094 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:52.088+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:52.161 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:52.156+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:52.180 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:52.174+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:52.377 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:52.372+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:52.398 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:52.392+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:52.586 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:52.580+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:53.099 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:53.090+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:53.125 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:53.118+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:53.159 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:53.153+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:53.333 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:53.328+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:53.443 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:53.433+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:53.636 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:53.632+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:54.095 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:54.089+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:54.134 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:54.128+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:54.163 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:54.158+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:54.343 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:54.337+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:54.409 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:54.404+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:54.622 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:54.618+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:55.125 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:55.120+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:55.159 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:55.154+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:55.169 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:55.163+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:55.349 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:55.343+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:55.425 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:55.420+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:55.588 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:55.582+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:56.114 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:56.108+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:56.201 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:56.196+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:56.202 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:56.196+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:56.352 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:56.346+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:56.400 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:56.395+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:56.632 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:56.627+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:57.071 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:57.067+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:57.188 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:57.183+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:57.209 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:57.203+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:57.347 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:57.342+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:57.443 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:57.437+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:57.605 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:57.595+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:58.061 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:58.057+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:58.219 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:58.213+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:58.247 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:58.242+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:58.335 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:58.331+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:58.413 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:58.408+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:58.603 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:58.594+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:51:58.603 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:58.596+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:51:58.878 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:51:58.789+0000 20e25700 -1 RGWWatcher::handle_error cookie 5421222912 err (107) Transport endpoint is not connected 2019-12-09T18:51:59.081 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:51:59.077+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:51:59.207 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:51:59.201+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:51:59.207 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:51:59.201+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:51:59.330 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:51:59.325+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:51:59.373 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:51:59.367+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:51:59.568 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:59.560+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:51:59.569 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:51:59.562+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:00.070 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:00.065+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:00.174 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:00.170+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:00.188 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:00.179+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:00.304 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:00.300+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:00.329 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:00.323+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:00.578 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:00.573+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:00.579 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:00.574+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:01.069 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:01.062+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:01.152 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:01.148+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:01.184 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:01.179+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:01.263 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:01.259+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:01.347 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:01.337+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:01.571 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:01.565+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:01.580 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:01.570+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:02.094 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:02.090+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:02.097 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:02.094+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:02.141 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:02.136+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:02.160 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:02.153+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:02.253 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:02.249+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:02.301 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:02.294+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:02.545 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:02.538+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:02.546 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:02.540+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:03.092 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:03.088+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:03.095 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:03.091+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:03.165 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:03.160+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:03.197 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:03.182+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:03.248 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:03.244+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:03.300 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:03.295+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:03.550 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:03.544+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:03.551 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:03.546+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:04.114 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:04.108+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:04.114 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:04.110+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:04.136 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:04.126+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:04.204 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:04.197+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:04.269 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:04.257+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:04.296 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:04.291+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:04.593 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:04.587+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:04.593 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:04.589+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:05.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:05.151+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:05.158 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:05.153+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:05.158 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:05.153+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:05.184 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:05.179+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:05.247 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:05.243+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:05.267 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:05.262+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:05.630 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:05.623+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:05.630 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:05.625+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:06.154 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:06.150+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:06.165 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:06.160+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:06.168 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:06.164+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:06.214 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:06.210+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:06.229 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:06.225+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:06.280 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:06.275+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:06.667 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:06.660+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:06.667 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:06.662+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:07.191 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:07.185+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:07.214 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:07.208+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:07.214 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:07.210+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:07.217 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:07.212+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:07.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:07.234+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:07.301 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:07.296+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:07.721 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:07.715+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:07.722 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:07.717+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:08.176 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:08.171+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:08.177 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:08.172+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:08.180 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:08.175+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:08.246 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:08.242+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:08.255 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:08.250+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:08.328 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:08.322+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:08.687 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:08.681+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:08.688 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:08.683+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:09.174 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:09.168+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:09.174 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:09.169+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:09.194 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:09.189+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:09.220 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:09.215+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:09.245 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:09.239+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:09.288 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:09.284+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:09.673 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:09.667+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:09.674 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:09.669+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:10.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:10.160+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:10.168 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:10.162+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:10.235 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:10.229+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:10.247 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:10.240+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:10.256 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:10.251+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:10.297 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:10.292+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:10.686 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:10.682+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:10.689 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:10.685+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:11.041 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:52:11.034+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e319 v319) 2019-12-09T18:52:11.175 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:11.169+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:11.175 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:11.171+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:11.214 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:11.208+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:11.242 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:11.238+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:11.298 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:11.292+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:11.318 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:11.313+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:11.684 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:11.677+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:11.686 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:11.681+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:12.179 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:12.174+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:12.218 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:12.211+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:12.219 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:12.175+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:12.219 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:12.205+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:12.291 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:12.286+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:12.356 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:12.351+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:12.679 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:12.672+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:12.680 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:12.674+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:13.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:13.158+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:13.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:13.160+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:13.185 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:13.180+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:13.219 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:13.213+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:13.328 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:13.322+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:13.380 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:13.374+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:13.721 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:13.716+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:13.727 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:13.719+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:14.200 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:14.194+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:14.201 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:14.196+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:14.204 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:14.198+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:14.255 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:14.247+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:14.336 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:14.331+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:14.383 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:14.378+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:14.715 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:14.708+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:14.715 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:14.710+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:15.204 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:15.196+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:15.210 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:15.198+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:15.240 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:15.234+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:15.277 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:15.262+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:15.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:15.345+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:15.388 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:15.383+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:15.703 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:15.698+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:15.705 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:15.698+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:16.062 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:52:16.054+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e319 v319) 2019-12-09T18:52:16.206 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:16.200+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:16.207 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:16.202+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:16.247 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:16.241+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:16.255 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:16.248+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:16.352 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:16.347+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:16.355 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:16.350+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:16.712 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:16.707+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:16.714 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:16.710+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:17.177 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:17.171+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:17.180 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:17.175+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:17.266 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:17.260+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:17.282 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:17.273+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:17.351 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:17.345+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:17.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:17.372+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:17.747 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:17.739+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:17.748 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:17.741+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:18.222 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:18.216+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:18.223 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:18.218+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:18.303 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:18.295+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:18.325 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:18.317+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:18.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:18.374+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:18.406 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:18.387+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:18.784 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:18.778+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:18.785 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:18.780+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:19.019 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:19.014+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:19.225 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:19.219+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:19.226 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:19.221+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:19.295 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:19.290+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:19.296 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:19.290+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:19.351 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:19.346+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:19.391 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:19.386+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:19.782 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:19.776+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:19.783 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:19.777+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:20.044 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:20.035+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:20.180 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:20.176+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:20.188 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:20.178+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:20.274 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:20.269+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:20.310 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:20.302+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:20.349 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:20.344+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:20.415 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:20.409+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:20.740 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:20.733+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:20.740 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:20.735+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:21.073 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:52:21.067+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e319 v319) 2019-12-09T18:52:21.095 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:21.091+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:21.161 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:21.155+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:21.162 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:21.158+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:21.239 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:21.234+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:21.301 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:21.295+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:21.371 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:21.365+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:21.477 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:21.473+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:21.711 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:21.702+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:21.711 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:21.704+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:22.115 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:22.108+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:22.200 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:22.195+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:22.203 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:22.199+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:22.279 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:22.274+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:22.353 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:22.344+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:22.375 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:22.369+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:22.481 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:22.472+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:22.721 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:22.713+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:22.722 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:22.715+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:23.092 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:23.088+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:23.213 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:23.207+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:23.213 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:23.208+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:23.327 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:23.320+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:23.329 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:23.323+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:23.424 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:23.417+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:23.528 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:23.520+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:23.761 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:23.754+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:23.763 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:23.756+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:24.066 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:24.061+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:24.268 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:24.261+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:24.269 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:24.264+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:24.372 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:24.367+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:24.378 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:24.372+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:24.418 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:24.412+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:24.525 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:24.518+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:24.755 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:24.748+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:24.756 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:24.749+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:25.101 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:25.096+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:25.252 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:25.245+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:25.252 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:25.247+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:25.385 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:25.380+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:25.412 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:25.404+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:25.444 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:25.437+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:25.538 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:25.531+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:25.712 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:25.705+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:25.712 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:25.707+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:26.099 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:52:26.094+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e319 v319) 2019-12-09T18:52:26.121 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:26.115+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:26.250 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:26.240+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:26.251 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:26.244+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:26.376 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:26.368+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:26.398 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:26.393+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:26.466 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:26.461+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:26.527 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:26.518+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:26.696 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:26.690+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:26.700 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:26.694+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:27.082 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:27.076+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:27.218 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:27.212+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:27.220 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:27.214+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:27.356 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:27.350+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:27.392 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:27.385+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:27.515 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:27.508+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:27.557 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:27.550+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:27.670 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:27.663+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:27.674 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:27.668+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:28.118 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:28.113+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:28.221 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:28.216+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:28.224 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:28.219+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:28.378 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:28.370+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:28.380 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:28.375+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:28.493 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:28.487+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:28.515 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:28.508+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:28.638 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:28.632+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:28.640 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:28.635+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:29.106 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:29.100+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:29.208 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:29.201+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:29.209 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:29.204+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:29.335 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:29.329+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:29.429 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:29.421+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:29.471 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:29.464+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:29.475 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:29.468+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:29.687 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:29.682+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:29.691 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:29.686+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:30.136 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:30.131+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:30.186 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:30.180+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:30.187 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:30.181+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:30.365 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:30.358+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:30.406 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:30.401+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:30.473 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:30.466+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:30.521 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:30.512+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:30.694 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:30.686+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:30.695 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:30.688+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:31.111 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:31.106+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:31.114 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:52:31.108+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e319 v319) 2019-12-09T18:52:31.236 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:31.230+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:31.237 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:31.231+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:31.386 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:31.379+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:31.401 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:31.396+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:31.498 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:31.489+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:31.513 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:31.502+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:31.688 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:31.672+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:31.689 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:31.673+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:32.093 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:32.087+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:32.255 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:32.250+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:32.260 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:32.255+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:32.401 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:32.394+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:32.402 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:32.397+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:32.464 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:32.457+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:32.542 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:32.536+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:32.652 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:32.646+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:32.656 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:32.651+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:33.093 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:33.088+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:33.313 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:33.306+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:33.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:33.308+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:33.368 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:33.361+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:33.443 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:33.438+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:33.501 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:33.495+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:33.579 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:33.572+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:33.622 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:33.615+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:33.623 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:33.617+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:34.094 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:34.089+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:34.281 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:34.273+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:34.281 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:34.277+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:34.352 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:34.346+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:34.436 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:34.430+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:34.541 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:34.533+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:34.575 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:34.568+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:34.677 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:34.668+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:34.678 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:34.670+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:35.103 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:35.098+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:35.281 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:35.275+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:35.282 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:35.277+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:35.393 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:52:35.387+0000 1694f700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.763006+0000 front 2019-12-09T18:50:56.947437+0000 (oldest deadline 2019-12-09T18:51:37.026446+0000) 2019-12-09T18:52:35.459 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:35.453+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:35.547 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:52:35.540+0000 1694f700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.290777+0000 front 2019-12-09T18:50:58.291671+0000 (oldest deadline 2019-12-09T18:51:41.844627+0000) 2019-12-09T18:52:35.566 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:35.560+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:35.721 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:35.714+0000 1694f700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:51:18.263482+0000 (oldest deadline 2019-12-09T18:51:58.263482+0000) 2019-12-09T18:52:35.722 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:35.715+0000 1694f700 -1 osd.3 319 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:36.152 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:52:36.147+0000 1694f700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.207659+0000 front 2019-12-09T18:50:56.004582+0000 (oldest deadline 2019-12-09T18:52:18.413983+0000) 2019-12-09T18:52:36.185 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:52:36.173+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 40sec e319 v319) 2019-12-09T18:52:36.308 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:36.303+0000 1694f700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:56.755649+0000 front 2019-12-09T18:50:56.773982+0000 (oldest deadline 2019-12-09T18:51:37.994696+0000) 2019-12-09T18:52:36.310 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:36.306+0000 1694f700 -1 osd.4 319 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:52:36.481 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:52:36.476+0000 1694f700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:57.213607+0000 front 2019-12-09T18:50:56.308882+0000 (oldest deadline 2019-12-09T18:51:40.446092+0000) 2019-12-09T18:52:36.580 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:52:36.574+0000 1694f700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:50:58.304800+0000 front 2019-12-09T18:50:58.305635+0000 (oldest deadline 2019-12-09T18:51:37.238956+0000) 2019-12-09T18:52:37.114 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:37.045+0000 1694f700 -1 osd.3 320 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:124414 2.2fs0 2.f7133eef (undecoded) ondisk+retry+read+known_if_redirected e317) 2019-12-09T18:52:37.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:52:37.162+0000 3b282700 -1 osd.3 320 build_incremental_map_msg missing incremental map 319 2019-12-09T18:52:37.438 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:52:37.428+0000 1694f700 -1 osd.4 320 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:124615 5.3 5:c609908c:::notify.5:head [watch ping cookie 5421222912] snapc 0=[] ondisk+write+known_if_redirected e319) 2019-12-09T18:53:01.978 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:53:01.965+0000 3aa81700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T18:53:02.006 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:53:01.990+0000 3aa81700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T18:53:02.051 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:53:01.921+0000 124b0700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T18:53:02.428 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:53:02.234+0000 35a77700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T18:54:22.302 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:22.296+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:22.975 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:22.965+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:23.295 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:23.289+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:24.004 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:23.998+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:24.182 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:24.175+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:24.322 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:24.311+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:24.897 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:24.891+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:24.903 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:24.897+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:25.040 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:25.032+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:25.160 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:25.154+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:25.276 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:25.270+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:25.464 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:25.457+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:25.854 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:25.848+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:25.872 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:25.866+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:26.004 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:25.998+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:26.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:26.128+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:26.250 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:26.245+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:26.321 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:26.315+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:26.505 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:26.499+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:26.816 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:26.810+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:26.909 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:26.903+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:27.024 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:27.017+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:27.192 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:27.184+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:27.243 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:27.236+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:27.302 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:27.294+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:27.547 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:27.540+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:27.824 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:27.818+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:27.956 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:27.948+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:28.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:28.059+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:28.202 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:28.196+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:28.211 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:28.205+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:28.316 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:28.310+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:28.558 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:28.553+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:28.816 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:28.810+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:28.987 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:28.979+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:29.035 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:29.028+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:29.221 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:29.214+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:29.251 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:29.244+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:29.370 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:29.361+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:29.606 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:29.600+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:29.862 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:29.855+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:30.008 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:30.002+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:30.041 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:30.035+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:30.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:30.192+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:30.238 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:30.232+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:30.336 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:30.329+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:30.636 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:30.628+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:30.906 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:30.900+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:30.981 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:30.976+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:31.077 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:31.071+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:31.207 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:31.202+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:31.293 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:31.286+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:31.372 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:31.367+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:31.664 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:31.659+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:31.944 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:31.938+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:31.988 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:31.982+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:32.116 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:32.110+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:32.191 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:32.185+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:32.313 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:32.307+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:32.364 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:32.353+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:32.715 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:32.708+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:32.918 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:32.912+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:33.035 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:33.029+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:33.102 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:33.096+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:33.200 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:33.194+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:33.307 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:33.301+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:33.353 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:33.348+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:33.741 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:33.734+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:33.916 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:33.910+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:34.027 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:34.021+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:34.086 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:34.081+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:34.184 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:34.179+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:34.265 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:34.258+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:34.333 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:34.326+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:34.782 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:34.776+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:34.885 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:34.878+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:35.013 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:35.007+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:35.125 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:35.119+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:35.174 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:35.170+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:35.235 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:35.229+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:35.344 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:35.338+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:35.784 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:35.777+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:35.857 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:35.850+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:36.025 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:36.019+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:36.126 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:36.119+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:36.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:36.129+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:36.208 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:36.202+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:36.345 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:36.339+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:36.799 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:36.793+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:36.849 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:36.843+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:37.072 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:37.066+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:37.089 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:37.083+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:37.179 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:37.174+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:37.247 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:37.241+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:37.352 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:37.346+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:37.805 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:37.799+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:37.826 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:37.819+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:38.033 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:38.027+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:38.070 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:38.065+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:38.148 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:38.142+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:38.288 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:38.282+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:38.393 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:38.387+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:38.783 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:38.778+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:38.801 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:38.795+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:39.008 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:39.003+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:39.070 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:39.064+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:39.125 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:39.119+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:39.263 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:39.258+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:39.435 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:39.429+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:39.761 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:39.756+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:39.763 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:39.756+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:39.989 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:39.984+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:40.088 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:40.082+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:40.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:40.091+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:40.253 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:40.247+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:40.413 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:40.408+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:40.791 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:40.785+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:40.793 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:40.787+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:40.972 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:40.965+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:41.078 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:41.073+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:41.124 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:41.118+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:41.290 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:41.284+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:41.432 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:41.427+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:41.825 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:41.819+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:41.849 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:41.843+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:41.955 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:41.950+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:42.052 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:42.045+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:42.170 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:42.163+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:42.265 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:42.259+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:42.399 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:42.394+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:42.826 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:42.821+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:42.888 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:42.882+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:42.980 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:42.973+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:43.077 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:43.071+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:43.206 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:43.200+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:43.221 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:43.215+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:43.379 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:43.374+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:43.838 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:43.833+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:43.859 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:43.854+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:44.021 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:44.016+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:44.073 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:44.066+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:44.174 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:44.168+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:44.221 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:44.216+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:44.341 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:44.336+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:44.819 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:44.812+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:44.892 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:44.886+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:45.035 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:45.029+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:45.078 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:45.072+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:45.159 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:45.153+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:45.228 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:45.222+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:45.386 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:45.380+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:45.866 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:45.860+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:45.947 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:45.940+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:45.992 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:45.986+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:46.130 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:46.125+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:46.152 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:46.144+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:46.213 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:46.207+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:46.347 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:46.341+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:46.826 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:46.820+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:46.923 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:46.916+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:46.952 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:46.945+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:47.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:47.089+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:47.164 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:47.158+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:47.232 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:47.226+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:47.306 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:47.301+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:47.843 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:47.837+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:47.919 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:47.912+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:47.963 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:47.957+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:48.138 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:48.133+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:48.205 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:48.197+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:48.249 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:48.243+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:48.311 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:48.305+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:48.803 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:48.798+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:48.955 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:48.948+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:48.967 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:48.962+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:49.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:49.127+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:49.213 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:49.206+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:49.270 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:49.264+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:49.270 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:49.263+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:49.811 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:49.805+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:49.930 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:49.923+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:49.949 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:49.943+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:50.162 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:50.157+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:50.212 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:50.206+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:50.314 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:50.307+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:50.315 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:50.310+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:50.859 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:50.853+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:50.947 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:50.941+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:50.950 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:50.944+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:51.149 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:51.144+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:51.232 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:51.226+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:51.279 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:51.273+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:51.331 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:51.326+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:51.823 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:51.817+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:51.941 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:51.934+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:51.979 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:51.974+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:52.152 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:52.146+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:52.205 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:52.199+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:52.278 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:52.272+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:52.349 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:52.343+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:52.796 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:52.790+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:52.980 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:52.974+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:53.025 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:53.017+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:53.108 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:53.103+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:53.233 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:53.224+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:53.233 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:53.227+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:53.386 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:53.380+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:53.835 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:53.829+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:54.004 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:53.999+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:54.038 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:54.029+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:54.088 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:54.083+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:54.203 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:54.197+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:54.280 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:54.274+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:54.406 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:54.399+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:54.867 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:54.861+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:55.047 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:55.041+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:55.074 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:55.068+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:55.133 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:55.127+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:55.215 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:55.209+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:55.259 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:55.253+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:55.420 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:55.412+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:55.902 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:55.896+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:56.012 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:56.007+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:56.088 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:56.081+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:56.132 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:56.127+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:56.215 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:56.209+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:56.263 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:56.256+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:56.422 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:56.416+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:56.596 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:54:56.590+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e325 v325) 2019-12-09T18:54:56.955 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:56.949+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:57.025 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:57.020+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:57.084 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:57.080+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:57.131 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:57.126+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:57.223 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:57.217+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:57.225 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:57.220+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:57.399 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:57.393+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:57.953 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:57.946+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:58.008 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:58.003+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:58.048 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:58.041+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:58.156 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:58.150+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:58.194 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:11:54.811 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.198 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:58.193+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:58.203 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:11:54.822 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.219 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:11:54.837 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.227 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:11:54.845 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.232 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:11:54.831 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.259 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:11:54.858 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.288 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:58.275+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:58.424 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:58.419+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:58.441 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:11:55.060 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.527 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:11:55.125 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.534 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:11:55.152 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:58.984 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:54:58.978+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:54:58.998 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:58.992+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:54:59.024 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:54:59.018+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:54:59.171 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:54:59.166+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:54:59.177 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:54:59.171+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:54:59.316 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:54:59.311+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:54:59.390 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:54:59.384+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:54:59.964 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:54:59.960+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:00.012 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:00.000+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:00.029 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:00.023+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:00.210 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:00.201+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:00.217 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:00.212+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:00.333 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:00.329+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:00.428 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:00.422+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:00.932 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:55:00.926+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:01.005 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:01.000+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:01.026 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:01.019+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:01.180 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:01.175+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:01.212 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:01.206+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:01.327 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:01.322+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:01.436 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:01.430+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:01.610 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:55:01.605+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e325 v325) 2019-12-09T18:55:01.965 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:55:01.954+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:01.981 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:01.977+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:01.986 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:01.978+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:02.215 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:02.211+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:02.223 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:02.217+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:02.326 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:02.322+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:02.426 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:02.419+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:02.959 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:02.954+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:02.991 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:55:02.985+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:03.035 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:03.028+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:03.205 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:03.200+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:03.281 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:03.266+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:03.328 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:03.323+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:03.453 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:03.446+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:03.949 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:03.943+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:03.977 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:55:03.972+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:04.065 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:04.059+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:04.261 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:04.257+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:04.277 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:04.270+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:04.325 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:04.320+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:04.478 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:04.471+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:04.969 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:04.964+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:05.005 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:55:05.000+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:05.070 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:05.062+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:05.229 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:05.224+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:05.294 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:05.289+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:05.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:05.299+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:05.499 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:05.493+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:05.963 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:55:05.957+0000 1694f700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:39.994478+0000 front 2019-12-09T18:53:40.309033+0000 (oldest deadline 2019-12-09T18:54:23.899189+0000) 2019-12-09T18:55:06.026 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:55:06.019+0000 1694f700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.368847+0000 front 2019-12-09T18:53:40.969890+0000 (oldest deadline 2019-12-09T18:54:24.954209+0000) 2019-12-09T18:55:06.042 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:55:06.035+0000 1694f700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:38.375667+0000 front 2019-12-09T18:53:41.329325+0000 (oldest deadline 2019-12-09T18:54:24.139408+0000) 2019-12-09T18:55:06.267 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:55:06.262+0000 1694f700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.326905+0000 front 2019-12-09T18:53:41.389227+0000 (oldest deadline 2019-12-09T18:54:25.893328+0000) 2019-12-09T18:55:06.289 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:55:06.283+0000 1694f700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312502+0000 front 2019-12-09T18:53:39.544836+0000 (oldest deadline 2019-12-09T18:54:23.905367+0000) 2019-12-09T18:55:06.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:55:06.308+0000 1694f700 -1 osd.4 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:40.312419+0000 front 2019-12-09T18:53:38.052780+0000 (oldest deadline 2019-12-09T18:54:22.136744+0000) 2019-12-09T18:55:06.524 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:55:06.516+0000 1694f700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:53:41.075474+0000 front 2019-12-09T18:53:41.387880+0000 (oldest deadline 2019-12-09T18:54:22.101972+0000) 2019-12-09T18:55:06.668 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:55:06.660+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e325 v325) 2019-12-09T18:55:44.629 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:44.620+0000 114ae700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:44.805 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:44.796+0000 3727a700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:44.958 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:44.818+0000 3a280700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.065 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:44.983+0000 36a79700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.073 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:44.995+0000 35276700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.157 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.058+0000 3827c700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.168 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.079+0000 11caf700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.185 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.106+0000 36a79700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.221 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.149+0000 36a79700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.262 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.163+0000 35a77700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.308 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.202+0000 35a77700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:45.429 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:45.230+0000 38a7d700 -1 osd.3 330 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:46.607 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:46.335+0000 3aa81700 -1 osd.3 331 build_incremental_map_msg missing incremental map 330 2019-12-09T18:55:49.441 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:55:49.411+0000 3727a700 -1 osd.3 331 build_incremental_map_msg missing incremental map 330 2019-12-09T18:56:05.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:05.208+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:05.244 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:05.238+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:05.879 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:05.873+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:06.147 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:06.140+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:06.170 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:06.163+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:06.205 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:06.200+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:06.889 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:06.884+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:07.113 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:07.107+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:07.201 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:07.196+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:07.233 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:07.228+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:07.861 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:07.855+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:08.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:08.045+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:08.097 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:08.091+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:08.166 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:08.161+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:08.231 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:08.226+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:08.304 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:08.298+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:08.837 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:08.831+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:09.088 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:09.083+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:09.101 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:09.094+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:09.199 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:09.194+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:09.236 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:09.231+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:09.270 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:09.264+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:09.845 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:09.840+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:10.094 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:10.088+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:10.154 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:10.149+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:10.155 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:10.150+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:10.248 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:10.242+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:10.289 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:10.284+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:10.872 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:10.866+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:11.142 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:11.137+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:11.172 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:11.165+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:11.194 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:11.188+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:11.208 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:11.203+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:11.257 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:11.252+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:11.923 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:11.914+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:12.117 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:12.111+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:12.128 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:12.123+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:12.168 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:12.163+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:12.207 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:12.201+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:12.303 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:12.297+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:12.948 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:12.942+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:13.103 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:13.098+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:13.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:13.109+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:13.155 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:13.151+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:13.239 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:13.234+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:13.313 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:13.307+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:13.916 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:13.911+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:14.137 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:14.132+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:14.138 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:14.133+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:14.143 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:14.138+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:14.143 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:14.140+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:14.211 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:14.207+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:14.297 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:14.291+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:14.929 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:14.923+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:15.097 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:15.092+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:15.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:15.128+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:15.172 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:15.167+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:15.182 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:15.178+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:15.199 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:15.194+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:15.280 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:15.275+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:15.960 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:15.954+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:16.058 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:16.052+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:16.160 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:16.156+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:16.166 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:16.161+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:16.169 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:16.164+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:16.178 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:16.169+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:16.298 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:16.293+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:16.968 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:16.961+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:17.096 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:17.090+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:17.176 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:17.171+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:17.196 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:17.190+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:17.198 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:17.192+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:17.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:17.208+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:17.322 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:17.316+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:17.937 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:17.932+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:18.093 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:18.088+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:18.172 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:18.169+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:18.206 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:18.196+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:18.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:18.241+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:18.247 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:18.242+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:18.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:18.340+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:18.986 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:18.981+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:19.109 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:19.102+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:19.169 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:19.163+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:19.178 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:19.172+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:19.251 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:19.245+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:19.269 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:19.262+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:19.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:19.340+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:19.976 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:19.970+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:20.132 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:20.126+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:20.134 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:20.127+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:20.134 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:20.129+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:20.148 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:20.143+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:20.217 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:20.211+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:20.272 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:20.263+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:20.322 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:20.314+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:21.014 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:21.007+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:21.096 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:21.090+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:21.171 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:21.160+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:21.171 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:21.161+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:21.189 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:21.183+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:21.215 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:21.209+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:21.266 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:21.259+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:21.384 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:21.375+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:22.045 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:22.039+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:22.114 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:22.109+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:22.152 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:22.147+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:22.210 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:22.204+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:22.211 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:22.205+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:22.242 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:22.237+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:22.275 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:22.269+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:22.367 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:22.361+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:23.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:23.065+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:23.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:23.097+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:23.120 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:23.114+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:23.213 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:23.207+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:23.217 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:23.211+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:23.239 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:23.234+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:23.303 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:23.297+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:23.365 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:23.360+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:24.065 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:24.060+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:24.105 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:24.100+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:24.117 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:24.106+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:24.196 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:24.191+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:24.210 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:24.203+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:24.211 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:24.204+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:24.291 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:24.285+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:24.325 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:24.319+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:25.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:25.067+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:25.131 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:25.122+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:25.153 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:25.147+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:25.196 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:25.190+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:25.247 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:25.239+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:25.247 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:25.241+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:25.342 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:25.336+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:25.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:25.341+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:26.050 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:26.043+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:26.158 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:26.153+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:26.160 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:26.154+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:26.187 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:26.182+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:26.246 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:26.240+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:26.247 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:26.241+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:26.335 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:26.331+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:26.364 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:26.357+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:27.056 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:27.049+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:27.128 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:27.123+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:27.140 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:27.137+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:27.161 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:27.155+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:27.187 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:13:23.806 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.204 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:13:23.823 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.235 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:27.228+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:27.236 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:27.230+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:27.244 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:13:23.863 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.256 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:13:23.875 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.276 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:13:23.875 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.282 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:13:23.881 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.344 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:27.337+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:27.355 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:13:23.974 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.384 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:27.379+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:27.397 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:13:24.015 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:27.413 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:13:24.012 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:56:28.086 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:28.078+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:28.110 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:28.107+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:28.137 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:28.127+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:28.175 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:28.171+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:28.243 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:28.237+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:28.244 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:28.238+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:28.298 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:28.292+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:28.366 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:28.362+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:28.571 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T18:56:28.559+0000 20e25700 -1 RGWWatcher::handle_error cookie 441208560 err (107) Transport endpoint is not connected 2019-12-09T18:56:29.071 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:29.063+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:29.094 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:29.088+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:29.140 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:29.134+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:29.217 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:29.211+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:29.223 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:29.214+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:29.223 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:29.217+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:29.338 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:29.331+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:29.413 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:29.408+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:30.062 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:30.058+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:30.132 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:30.126+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:30.171 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:30.165+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:30.207 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:30.201+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:30.252 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:30.242+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:30.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:30.245+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:30.335 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:30.328+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:30.449 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:30.441+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:31.069 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:31.065+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:31.105 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:31.098+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:31.127 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:31.120+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:31.239 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:31.235+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:31.288 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:31.278+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:31.289 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:31.280+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:31.333 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:31.327+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:31.415 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:31.411+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:32.088 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:32.084+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:32.091 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:32.088+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:32.113 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:32.105+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:32.114 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:32.107+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:32.234 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:32.230+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:32.283 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:32.277+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:32.283 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:32.278+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:32.340 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:32.333+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:32.424 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:32.420+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:33.068 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:33.061+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:33.118 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:33.112+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:33.126 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:33.122+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:33.130 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:33.126+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:33.251 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:33.245+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:33.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:33.248+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:33.274 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:33.269+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:33.295 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:33.290+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:33.382 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:33.378+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:34.092 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:34.086+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:34.130 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:34.125+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:34.131 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:34.127+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:34.165 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:34.160+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:34.276 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:34.268+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:34.287 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:34.281+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:34.290 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:34.285+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:34.315 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:34.312+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:34.372 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:34.366+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:35.139 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:35.132+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:35.166 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:35.161+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:35.167 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:35.162+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:35.184 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:35.178+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:35.234 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:35.227+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:35.290 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:35.285+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:35.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:35.289+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:35.297 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:35.291+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:35.415 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:35.410+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:36.136 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:36.130+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:36.137 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:36.131+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:36.190 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:36.184+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:36.201 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:36.193+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:36.229 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:36.223+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:36.257 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:36.249+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:36.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:36.342+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:36.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:36.344+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:36.421 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:36.416+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:37.096 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:37.091+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:37.099 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:37.095+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:37.156 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:37.151+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:37.191 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:37.185+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:37.207 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:37.201+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:37.239 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:56:37.234+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 39sec e331 v331) 2019-12-09T18:56:37.300 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:37.294+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:37.311 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:37.306+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:37.313 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:37.308+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:37.408 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:37.402+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:38.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:38.060+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:38.067 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:38.062+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:38.145 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:38.138+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:38.169 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:38.156+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:38.244 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:38.238+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:38.292 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:38.287+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:38.295 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:38.291+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:38.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:38.290+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:38.403 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:38.397+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:39.056 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:39.051+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:39.059 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:39.055+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:39.132 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:39.127+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:39.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:39.192+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:39.252 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:39.246+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:39.292 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:39.285+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:39.293 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:39.286+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:39.338 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:39.334+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:39.391 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:39.386+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:40.082 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:40.077+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:40.086 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:40.081+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:40.184 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:40.178+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:40.229 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:40.223+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:40.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:40.245+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:40.281 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:40.276+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:40.283 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:40.278+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:40.351 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:40.345+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:40.367 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:40.361+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:41.081 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:41.076+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:41.082 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:41.077+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:41.168 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:41.161+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:41.202 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:41.195+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:41.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:41.239+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:41.323 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:41.316+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:41.324 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:41.318+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:41.382 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:41.376+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:41.392 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:41.387+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:42.055 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:42.049+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:42.055 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:42.050+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:42.181 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:42.174+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:42.195 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:42.189+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:42.265 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:56:42.257+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 39sec e331 v331) 2019-12-09T18:56:42.267 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:42.262+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:42.289 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:42.283+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:42.291 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:42.286+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:42.358 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:42.350+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:42.361 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:42.357+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:43.046 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:43.040+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:43.046 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:43.042+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:43.173 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:43.166+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:43.209 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:43.202+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:43.229 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:43.223+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:43.299 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:43.292+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:43.300 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:43.293+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:43.360 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:43.353+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:43.375 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:43.368+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:44.032 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:44.027+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:44.034 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:44.030+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:44.208 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:44.203+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:44.216 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:44.210+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:44.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:44.226+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:44.260 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:44.254+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:44.263 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:44.258+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:44.330 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:44.325+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:44.396 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:44.390+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:44.998 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:44.990+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:44.998 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:44.992+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:45.183 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:45.176+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:45.204 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:45.197+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:45.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:45.241+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:45.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:45.243+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:45.266 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:45.260+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:45.371 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:45.365+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:45.376 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:45.371+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:45.969 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:45.963+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:45.970 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:45.965+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:46.199 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:46.193+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:46.208 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:46.202+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:46.209 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:46.202+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:46.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:46.226+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:46.237 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:46.230+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:46.409 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:46.403+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:46.420 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:46.415+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:46.961 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:46.956+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:46.964 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:46.960+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:47.179 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:47.173+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:47.233 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:47.227+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:47.236 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:47.229+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:47.236 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:47.231+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:47.271 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:47.263+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:47.288 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:56:47.278+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 39sec e331 v331) 2019-12-09T18:56:47.427 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:47.421+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:47.468 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:47.464+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:47.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:47.927+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:47.936 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:47.931+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:48.141 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:48.135+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:48.222 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:48.215+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:48.223 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:48.217+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:48.249 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:48.243+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:48.253 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:48.247+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:48.458 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:48.453+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:48.479 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:48.470+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:48.986 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:48.981+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:48.989 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:48.985+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:49.146 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:49.139+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:49.215 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:49.208+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:49.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:49.225+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:49.254 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:49.247+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:49.263 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:49.249+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:49.473 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:49.468+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:49.484 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:49.479+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:49.956 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:49.950+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:49.956 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:49.951+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:50.114 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:50.109+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:50.204 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:50.198+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:50.215 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:50.209+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:50.218 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:50.213+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:50.257 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:50.251+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:50.445 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:50.440+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:50.504 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:50.497+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:50.924 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:50.918+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:50.925 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:50.920+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:51.110 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:51.103+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:51.231 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:51.225+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:51.249 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:51.244+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:51.257 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:51.246+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:51.257 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:51.252+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:51.423 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:51.417+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:51.517 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:51.506+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:51.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:51.927+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:51.935 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:51.931+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:52.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:52.069+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:52.214 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:52.208+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:52.221 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:52.215+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:52.224 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:52.218+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:52.225 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:52.219+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:52.297 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:56:52.290+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 39sec e331 v331) 2019-12-09T18:56:52.406 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:52.401+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:52.538 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:52.533+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:52.910 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:52.905+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:52.911 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:52.906+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:53.123 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:53.117+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:53.182 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:53.176+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:53.184 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:53.178+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:53.187 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:53.182+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:53.244 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:53.238+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:53.447 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:53.442+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:53.494 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:53.489+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:53.926 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:53.920+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:53.927 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:53.922+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:54.084 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:54.077+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:54.192 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:54.180+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:54.197 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:54.186+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:54.197 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:54.189+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:54.235 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:54.230+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:54.485 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:54.479+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:54.502 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:54.496+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:54.889 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:54.884+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:54.890 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:54.885+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:55.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:55.069+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:55.169 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:55.163+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:55.179 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:55.170+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:55.180 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:55.172+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:55.222 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:55.215+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:55.492 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:55.487+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:55.513 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:55.507+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:55.886 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:55.879+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:55.887 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:55.881+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:56.063 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:56.056+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:56.163 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:56.157+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:56.164 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:56.158+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:56.198 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:56.192+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:56.276 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:56.270+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:56.476 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:56.470+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:56.508 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:56:56.503+0000 1694f700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.167526+0000 front 2019-12-09T18:55:21.884879+0000 (oldest deadline 2019-12-09T18:56:05.953770+0000) 2019-12-09T18:56:56.870 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:56.864+0000 1694f700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:22.092472+0000 front 2019-12-09T18:55:22.566691+0000 (oldest deadline 2019-12-09T18:56:07.024466+0000) 2019-12-09T18:56:56.871 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:56.866+0000 1694f700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:57.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:56:57.049+0000 1694f700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.323053+0000 front 2019-12-09T18:55:25.313923+0000 (oldest deadline 2019-12-09T18:56:04.336109+0000) 2019-12-09T18:56:57.124 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:57.116+0000 1694f700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T18:55:39.916684+0000 (oldest deadline 2019-12-09T18:56:19.916684+0000) 2019-12-09T18:56:57.124 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:57.118+0000 1694f700 -1 osd.3 331 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:57.194 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:56:57.187+0000 1694f700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:25.375651+0000 front 2019-12-09T18:55:25.373845+0000 (oldest deadline 2019-12-09T18:56:04.902958+0000) 2019-12-09T18:56:57.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:56:57.299+0000 1694f700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:21.167896+0000 front 2019-12-09T18:55:20.092866+0000 (oldest deadline 2019-12-09T18:56:04.766506+0000) 2019-12-09T18:56:57.385 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:56:57.380+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 39sec e331 v331) 2019-12-09T18:56:57.481 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:56:57.476+0000 1694f700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:55:24.429610+0000 front 2019-12-09T18:55:24.428459+0000 (oldest deadline 2019-12-09T18:56:07.795284+0000) 2019-12-09T18:56:57.967 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:57.963+0000 1694f700 -1 osd.4 332 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:56:58.255 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T18:56:58.204+0000 1694f700 -1 osd.3 332 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-12-09T18:56:59.095 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:56:59.074+0000 1694f700 -1 osd.4 333 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:125060 5.3 5:c609908c:::notify.5:head [watch ping cookie 441208560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e331) 2019-12-09T18:58:42.961 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:42.957+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:43.953 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:43.948+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:45.005 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:45.000+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:46.033 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:46.028+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:47.012 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:47.006+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:48.015 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:48.010+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:49.003 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:48.999+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:50.000 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:49.996+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:50.958 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:50.954+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:51.989 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:51.983+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:51.990 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:51.986+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:52.963 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:15:49.582 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:52.971 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:52.965+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:52.971 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:52.967+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:52.979 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:15:49.598 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:52.989 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:15:49.608 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:52.996 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:15:49.615 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:52.998 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:52.992+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:52.998 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:15:49.597 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.009 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:15:49.608 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.102 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:53.095+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:53.157 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:15:49.776 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.177 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:15:49.796 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.217 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:15:49.816 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.356 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:53.351+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:53.558 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:53.550+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:53.810 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:53.806+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:53.812 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:53.806+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:53.984 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:53.977+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:53.985 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:53.979+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:54.043 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:54.038+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:54.099 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:54.094+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:54.386 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:54.381+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:54.600 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:54.595+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:54.813 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:54.808+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:54.847 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:54.841+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:54.984 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:54.979+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:54.984 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:54.980+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:55.004 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:54.999+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:55.098 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:55.093+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:55.405 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:55.399+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:55.621 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:55.615+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:55.838 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:55.833+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:55.848 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:55.841+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:55.967 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:55.960+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:55.968 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:55.963+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:56.015 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:56.010+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:56.100 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:56.095+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:56.418 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:56.412+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:56.656 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:56.650+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:56.816 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:56.811+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:56.824 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:56.811+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:56.967 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:56.962+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:56.968 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:56.963+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:57.039 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:57.033+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:57.108 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:57.102+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:57.471 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:57.463+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:57.703 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:57.697+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:57.820 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:57.813+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:57.848 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:57.842+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:57.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:57.936+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:57.942 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:57.939+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:58.073 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:58.068+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:58.117 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:58.111+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:58.510 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:58.503+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:58.729 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:58.723+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:58.788 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:58.782+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:58.820 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:58.814+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:58.971 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:58.965+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:58.972 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:58.967+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:58:59.104 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:58:59.099+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:58:59.117 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:58:59.103+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:58:59.508 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:58:59.501+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:58:59.731 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:58:59.725+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:58:59.796 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:58:59.790+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:58:59.817 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:58:59.811+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:58:59.931 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:59.924+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:58:59.933 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:58:59.926+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:00.138 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:00.130+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:00.157 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:00.150+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:00.502 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:00.497+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:00.718 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:00.713+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:00.788 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:00.780+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:00.812 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:00.805+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:00.961 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:00.954+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:00.961 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:00.956+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:01.120 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:01.115+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:01.165 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:01.158+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:01.559 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:01.552+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:01.715 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:01.708+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:01.772 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:01.766+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:01.839 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:01.832+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:01.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:01.934+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:01.942 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:01.936+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:02.094 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:02.086+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:02.136 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:02.130+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:02.530 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:02.523+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:02.714 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:02.708+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:02.778 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:02.772+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:02.885 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:02.878+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:02.934 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:02.928+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:02.937 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:02.932+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:03.071 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:03.063+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:03.139 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:03.133+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:03.565 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:03.559+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:03.696 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:03.691+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:03.803 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:03.798+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:03.851 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:03.845+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:03.964 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:03.956+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:03.965 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:03.959+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:04.046 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:04.040+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:04.194 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:04.188+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:04.600 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:04.594+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:04.752 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:04.745+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:04.783 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:04.777+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:04.819 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:04.813+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:05.003 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:04.997+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:05.006 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:05.000+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:05.104 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:05.095+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:05.245 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:05.238+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:05.601 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:05.595+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:05.747 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:05.740+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:05.802 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:05.796+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:05.823 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:05.817+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:06.044 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:06.036+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:06.045 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:06.039+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:06.143 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:06.137+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:06.288 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:06.282+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:06.573 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:06.568+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:06.705 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:06.699+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:06.785 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:06.779+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:06.855 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:06.849+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:07.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:07.088+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:07.097 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:07.091+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:07.106 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:07.100+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:07.333 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:07.326+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:07.598 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:07.593+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:07.708 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:07.702+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:07.809 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:07.802+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:07.895 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:07.889+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:08.092 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:08.086+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:08.095 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:08.090+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:08.145 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:08.139+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:08.334 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:08.327+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:08.563 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:08.557+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:08.690 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:08.684+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:08.860 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:08.853+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:08.901 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:08.885+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:09.141 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:09.134+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:09.141 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:09.136+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:09.200 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:09.193+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:09.387 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:09.380+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:09.616 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:09.609+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:09.677 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:09.671+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:09.815 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:09.808+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:09.854 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:09.849+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:10.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:10.129+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:10.136 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:10.131+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:10.179 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:10.173+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:10.397 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:10.390+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:10.570 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:10.564+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:10.724 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:10.716+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:10.781 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:10.773+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:10.869 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:10.862+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:11.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:11.108+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:11.119 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:11.113+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:11.199 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:11.192+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:11.400 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:11.395+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:11.567 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:11.561+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:11.736 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:11.730+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:11.751 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:11.745+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:11.901 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:11.895+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:12.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:12.078+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:12.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:12.079+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:12.248 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:12.243+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:12.375 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:12.368+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:12.548 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:12.541+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:12.707 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:12.699+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:12.792 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:12.785+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:12.946 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:12.940+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:13.117 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:13.110+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:13.118 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:13.112+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:13.307 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:13.302+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:13.356 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:13.350+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:13.515 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:13.509+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:13.683 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:13.676+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:13.820 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:13.814+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:13.998 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:13.993+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:14.149 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:14.142+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:14.151 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:14.146+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:14.274 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:14.268+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:14.312 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:14.306+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:14.571 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:14.561+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:14.736 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:14.728+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:14.775 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:14.769+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:14.976 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:14.970+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:15.164 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:15.156+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:15.167 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:15.161+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:15.306 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:15.300+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:15.352 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:15.346+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:15.542 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:15.535+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:15.744 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:15.736+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:15.776 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:15.770+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:15.980 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:15.973+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:16.141 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:16.134+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:16.143 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:16.136+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:16.362 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:16.357+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:16.363 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:16.357+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:16.503 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:16.498+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:16.761 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:16.754+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:16.772 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:16.766+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:16.950 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:16.943+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:17.144 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:17.137+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:17.147 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:17.142+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:17.356 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:17.351+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:17.391 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:17.385+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:17.530 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:17.525+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:17.732 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:17.726+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:17.799 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:17.793+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:17.995 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:17.989+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:18.184 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:18.178+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:18.184 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:18.178+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:18.361 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:18.355+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:18.404 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:18.397+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:18.547 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:18.541+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:18.733 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:18.727+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:18.818 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:18.812+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:19.008 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:19.000+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:19.138 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:19.131+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:19.139 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:19.133+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:19.375 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:19.368+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:19.396 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:19.389+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:19.603 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:19.597+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:19.780 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:19.773+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:19.809 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:19.803+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:20.047 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:20.042+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:20.121 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:20.115+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:20.123 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:20.118+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:20.378 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:20.373+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:20.390 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:20.383+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:20.655 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:20.649+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:20.746 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:20.739+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:20.834 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:20.828+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:21.008 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:21.001+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:21.170 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:21.163+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:21.171 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:21.165+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:21.356 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:21.350+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:21.377 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:21.371+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:21.695 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:21.687+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:21.774 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:21.765+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:21.821 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:21.812+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:22.037 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:22.030+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:22.159 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:22.153+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:22.160 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:22.154+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:22.332 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:22.325+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:22.402 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:22.396+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:22.742 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:22.736+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:22.788 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:22.781+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:22.814 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:22.808+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:22.854 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:59:22.846+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e337 v337) 2019-12-09T18:59:23.022 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:23.016+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:23.122 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:23.115+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:23.123 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:23.116+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:23.382 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:23.377+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:23.397 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:23.391+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:23.759 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:23.751+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:23.768 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:23.762+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:23.851 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:23.846+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:24.000 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:23.994+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:24.086 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:24.080+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:24.089 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:24.083+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:24.370 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:24.363+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:24.383 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:24.378+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:24.749 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:24.740+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:24.814 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:24.808+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:24.878 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:24.870+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:25.035 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:25.030+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:25.124 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:25.116+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:25.126 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:25.120+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:25.405 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:25.396+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:25.412 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:25.407+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:25.802 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:25.790+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:25.862 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:25.853+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:25.928 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:25.922+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:26.065 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:26.056+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:26.081 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:26.074+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:26.084 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:26.079+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:26.379 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:26.374+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:26.447 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:26.441+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:26.808 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:26.799+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:26.828 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:26.823+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:26.976 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:26.971+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:27.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:27.046+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:27.109 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:27.103+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:27.112 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:27.106+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:27.370 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:27.366+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:27.447 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:27.441+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:27.833 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:27.826+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:27.840 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:27.829+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:27.875 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:59:27.868+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e337 v337) 2019-12-09T18:59:27.933 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:27.928+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:28.076 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:28.072+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:28.125 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:28.118+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:28.127 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:28.123+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:28.371 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:28.365+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:28.462 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:28.454+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:28.836 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:28.830+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:28.867 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:28.862+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:28.979 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:28.974+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:29.057 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:29.051+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:29.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:29.077+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:29.085 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:29.079+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:29.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:29.414+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:29.458 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:29.451+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:29.799 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:29.793+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:29.893 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:29.883+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:29.997 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:29.993+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:30.086 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:30.080+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:30.104 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:30.089+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:30.104 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:30.090+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:30.467 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:30.461+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:30.503 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:30.496+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:30.813 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:30.804+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:30.942 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:30.937+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:30.981 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:30.975+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:31.052 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:31.046+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:31.115 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:31.109+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:31.117 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:31.110+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:31.464 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:31.457+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:31.515 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:31.510+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:31.806 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:31.799+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:31.973 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:31.968+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:31.985 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:31.979+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:32.080 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:32.072+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:32.080 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:32.075+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:32.093 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T18:59:32.087+0000 1694f700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:09.294619+0000 front 2019-12-09T18:58:09.233965+0000 (oldest deadline 2019-12-09T18:58:53.182191+0000) 2019-12-09T18:59:32.465 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T18:59:32.457+0000 1694f700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.683740+0000 front 2019-12-09T18:58:08.700551+0000 (oldest deadline 2019-12-09T18:58:52.477671+0000) 2019-12-09T18:59:32.494 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:32.489+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:32.777 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T18:59:32.765+0000 1694f700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.441431+0000 front 2019-12-09T18:58:08.278375+0000 (oldest deadline 2019-12-09T18:58:52.934250+0000) 2019-12-09T18:59:32.945 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T18:59:32.939+0000 1694f700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.483379+0000 front 2019-12-09T18:58:08.485339+0000 (oldest deadline 2019-12-09T18:58:52.467107+0000) 2019-12-09T18:59:32.953 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T18:59:32.948+0000 1694f700 -1 osd.6 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:08.744973+0000 front 2019-12-09T18:58:08.734662+0000 (oldest deadline 2019-12-09T18:58:53.030853+0000) 2019-12-09T18:59:32.983 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T18:59:32.974+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 44sec e337 v337) 2019-12-09T18:59:33.062 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:33.054+0000 1694f700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:07.541571+0000 front 2019-12-09T18:58:08.730837+0000 (oldest deadline 2019-12-09T18:58:51.608829+0000) 2019-12-09T18:59:33.062 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:33.057+0000 1694f700 -1 osd.0 337 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] ondisk+write+known_if_redirected e337) 2019-12-09T18:59:33.471 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T18:59:33.468+0000 1694f700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T18:58:10.455641+0000 front 2019-12-09T18:58:10.439985+0000 (oldest deadline 2019-12-09T18:58:52.086604+0000) 2019-12-09T18:59:34.456 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:34.333+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e337) 2019-12-09T18:59:35.510 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T18:59:35.504+0000 1694f700 -1 osd.0 339 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125324 3.3f 3:ff020696:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.622:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e337) 2019-12-09T19:00:03.742 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:03.647+0000 1694f700 -1 osd.4 340 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:125455 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 821443264 gen 117] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-09T19:00:27.294 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:00:27.288+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:00:27.261975+0000) 2019-12-09T19:00:28.296 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:00:28.291+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:00:27.261975+0000) 2019-12-09T19:00:29.271 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:00:29.265+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:00:27.261975+0000) 2019-12-09T19:00:31.403 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:31.398+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:32.391 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:32.385+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:32.783 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:32.777+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:33.404 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:33.399+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:33.756 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:33.749+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:34.195 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:34.190+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:34.360 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:34.355+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:34.363 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:34.357+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:34.699 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:34.692+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:34.762 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:34.756+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:35.166 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:35.161+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:35.400 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:35.392+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:35.413 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:35.407+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:35.721 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:35.714+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:35.742 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:35.735+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:36.133 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:36.123+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:36.452 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:36.447+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:36.453 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:36.447+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:36.714 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:36.709+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:36.752 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:36.747+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:37.130 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:37.124+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:37.422 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:37.416+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:37.453 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:37.446+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:37.737 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:37.731+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:37.756 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:37.750+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:38.121 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:38.116+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:38.432 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:38.426+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:38.454 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:38.448+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:38.759 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:38.754+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:38.803 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:38.797+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:39.081 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:39.076+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:39.436 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:39.429+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:39.454 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:39.448+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:39.782 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:39.776+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:39.804 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:39.799+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:40.122 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:40.117+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:40.452 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:40.445+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:40.464 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:40.458+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:40.762 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:40.756+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:40.768 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:40.762+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:41.152 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:41.148+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:41.438 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:41.432+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:41.490 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:41.485+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:41.755 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:41.745+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:41.790 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:41.784+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:42.206 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:42.201+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:42.468 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:42.463+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:42.469 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:42.461+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:42.750 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:42.744+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:42.789 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:42.783+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:43.260 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:43.254+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:43.510 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:43.504+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:43.521 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:43.515+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:43.773 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:43.766+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:43.793 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:43.786+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:44.274 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:44.269+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:44.483 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:44.477+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:44.515 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:44.509+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:44.788 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:44.782+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:44.816 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:44.807+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:44.838 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:44.833+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:45.255 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:45.246+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:45.508 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:45.501+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:45.515 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:45.508+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:45.785 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:45.779+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:45.795 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:45.788+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:45.837 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:45.831+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:46.278 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:46.273+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:46.488 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:46.482+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:46.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:46.489+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:46.802 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:46.795+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:46.827 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:46.821+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:46.872 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:46.866+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:47.309 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:00:47.304+0000 1694f700 -1 osd.7 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.128169+0000 front 2019-12-09T18:59:51.079970+0000 (oldest deadline 2019-12-09T19:00:33.412175+0000) 2019-12-09T19:00:47.458 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:47.453+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:47.521 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:47.515+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:47.797 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:47.790+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:47.840 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:47.830+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:47.907 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:47.902+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:48.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:48.458+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:48.475 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:48.470+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:48.788 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:48.782+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:48.816 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:48.807+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:48.895 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:48.890+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:49.446 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:49.440+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:49.528 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:49.521+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:49.750 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:49.744+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:49.865 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:49.859+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:49.911 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:49.904+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:50.490 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:50.484+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:50.490 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:50.484+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:50.725 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:50.720+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:50.839 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:50.832+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:50.919 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:50.912+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:51.470 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:51.464+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:51.499 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:51.493+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:51.753 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:51.746+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:51.877 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:51.870+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:51.934 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:51.928+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:51.937 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:51.932+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:52.470 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:52.466+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:52.535 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:52.529+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:52.773 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:52.768+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:52.906 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:52.899+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:52.933 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:52.927+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:52.935 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:52.930+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:53.447 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:53.439+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:53.524 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:53.519+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:53.829 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:53.820+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:53.884 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:53.878+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:53.952 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:53.944+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:53.953 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:53.947+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:54.473 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:54.468+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:54.532 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:54.526+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:54.809 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:54.801+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:54.844 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:54.838+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:54.912 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:54.906+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:54.914 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:54.909+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:55.440 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:55.435+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:55.495 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:55.488+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:55.806 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:55.800+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:55.885 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:55.878+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:55.930 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:55.924+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:55.934 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:55.929+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:56.398 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:56.393+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:56.508 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:56.502+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:56.770 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:56.763+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:56.856 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:56.850+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:56.961 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:56.954+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:56.962 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:56.956+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:57.445 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:57.440+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:57.559 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:57.551+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:57.761 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:57.754+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:57.905 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:57.899+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:57.957 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:57.948+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:57.958 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:57.950+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:58.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:58.457+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:58.547 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:58.540+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:58.805 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:58.795+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:58.916 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:58.910+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:58.919 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:58.913+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:58.938 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:58.929+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:00:59.491 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:00:59.487+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:00:59.572 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:00:59.566+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:00:59.795 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:00:59.789+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:00:59.907 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:59.900+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:00:59.907 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:00:59.902+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:00:59.910 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:00:59.904+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:00.486 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:00.482+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:00.537 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:00.531+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:00.785 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:00.779+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:00.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:00.937+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:00.947 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:00.941+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:00.947 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:00.942+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:01.498 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:01.494+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:01.508 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:01.501+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:01.791 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:01.785+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:01.917 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:01.908+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:01.917 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:01.910+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:01.950 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:01.945+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:02.519 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:02.512+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:02.530 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:02.524+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:02.835 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:02.829+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:02.920 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:02.911+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:02.921 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:02.914+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:03.005 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:02.996+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:03.324 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:01:03.317+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e344 v344) 2019-12-09T19:01:03.517 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:03.512+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:03.525 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:03.522+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:03.807 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:03.800+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:03.968 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:03.963+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:03.970 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:03.965+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:03.983 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:03.975+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:04.488 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:04.484+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:04.530 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:04.522+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:04.788 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:04.782+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:04.939 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:04.933+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:04.949 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:04.938+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:04.950 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:04.942+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:05.537 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:05.531+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:05.552 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:05.544+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:05.819 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:05.811+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:05.921 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:05.914+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:05.923 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:05.916+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:05.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:05.966+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:06.576 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:06.569+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:06.596 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:06.587+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:06.821 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:06.813+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:06.921 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:06.911+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:06.921 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:06.913+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:06.988 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:06.981+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:07.546 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:07.540+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:07.624 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:07.618+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:07.791 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:07.785+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:07.917 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:07.908+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:07.917 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:07.910+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:08.036 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:08.028+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:08.334 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:01:08.328+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e344 v344) 2019-12-09T19:01:08.599 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:08.594+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:08.622 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:08.616+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:08.836 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:08.828+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:08.958 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:08.951+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:08.958 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:08.953+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:09.011 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:09.005+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:09.603 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:09.596+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:09.658 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:09.652+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:09.813 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:09.804+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:09.972 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:09.965+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:09.975 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:09.967+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:10.043 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:10.033+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:10.636 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:10.629+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:10.636 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:10.618+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:10.792 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:10.786+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:10.955 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:10.948+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:10.957 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:10.950+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:11.008 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:11.003+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:11.637 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:11.632+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:11.651 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:11.645+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:11.835 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:11.826+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:11.966 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:11.946+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:11.966 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:11.948+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:12.065 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:12.057+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:12.613 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:12.608+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:12.622 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:12.616+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:12.875 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:12.870+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:12.972 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:12.967+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:12.973 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:12.968+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:13.060 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:13.054+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:13.352 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:01:13.345+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e344 v344) 2019-12-09T19:01:13.636 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:13.630+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:13.652 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:13.644+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:13.925 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:13.916+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:13.966 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:13.958+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:13.967 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:13.960+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:14.013 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:14.008+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:14.612 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:14.607+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:14.671 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:14.666+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:14.889 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:14.883+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:14.982 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:14.975+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:14.983 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:14.977+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:14.989 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:14.984+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:15.598 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:15.593+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:15.652 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:15.647+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:15.913 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:15.905+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:15.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:15.950+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:15.960 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:15.951+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:16.003 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:15.998+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:16.579 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:16.568+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:16.648 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:16.642+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:16.664 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:16.659+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:16.881 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:16.876+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:16.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:16.969+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:16.982 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:16.977+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:16.989 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:16.981+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:17.605 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:17.601+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:17.657 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:17.652+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:17.718 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:17.712+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:17.882 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:17.876+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:17.964 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:17.959+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:17.968 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:17.963+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:17.995 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:17.988+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:18.366 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:01:18.360+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e344 v344) 2019-12-09T19:01:18.592 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:18.585+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:18.681 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:18.675+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:18.767 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:18.761+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:18.901 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:18.895+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:18.972 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:18.964+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:18.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:18.988+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:18.996 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:18.990+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:19.579 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:19.571+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:19.680 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:19.674+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:19.790 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:19.784+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:19.933 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:19.927+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:19.943 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:19.939+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:19.981 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:19.975+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:19.981 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:19.976+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:20.550 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:20.545+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:20.715 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:20.710+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:20.757 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:20.752+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:20.920 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:20.914+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:20.980 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:20.974+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:21.035 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:21.018+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:21.035 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:21.019+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:21.536 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:21.531+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:21.757 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:21.747+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:21.809 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:21.802+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:21.885 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:21.879+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:21.996 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:21.990+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:21.997 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:21.992+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:22.008 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:22.002+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:22.571 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:22.566+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:22.768 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:22.764+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:22.800 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:22.795+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:22.900 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:22.894+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:22.985 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:22.978+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:22.985 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:22.980+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:22.994 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:22.988+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:23.384 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:01:23.376+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e344 v344) 2019-12-09T19:01:23.620 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:23.613+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:23.757 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:23.751+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:23.772 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:23.766+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:23.892 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:23.887+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:23.977 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:23.971+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:24.013 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:24.008+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:24.018 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:24.012+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:24.582 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:24.577+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:24.763 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:24.758+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:24.797 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:24.791+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:24.934 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:24.928+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:25.005 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:24.996+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:25.006 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:25.000+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:25.009 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:25.003+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:25.553 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:25.547+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:25.736 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:25.729+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:25.767 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:25.761+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:25.958 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:25.953+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:25.969 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:25.959+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:25.970 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:25.961+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:26.006 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:26.001+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:26.535 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:26.530+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:26.697 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:26.692+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:26.753 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:26.744+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:26.966 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:26.960+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:26.978 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:26.973+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:27.006 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:27.000+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:27.007 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:27.002+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:27.586 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:27.581+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:27.696 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:01:27.690+0000 1694f700 -1 osd.2 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.663024+0000 front 2019-12-09T18:59:51.056947+0000 (oldest deadline 2019-12-09T19:00:33.921631+0000) 2019-12-09T19:01:27.746 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:27.740+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:27.969 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:01:27.963+0000 1694f700 -1 osd.1 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.628439+0000 front 2019-12-09T18:59:50.789127+0000 (oldest deadline 2019-12-09T19:00:34.431738+0000) 2019-12-09T19:01:27.978 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:01:27.973+0000 1694f700 -1 osd.0 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.230554+0000 front 2019-12-09T18:59:50.595597+0000 (oldest deadline 2019-12-09T19:00:31.768963+0000) 2019-12-09T19:01:28.026 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:28.021+0000 1694f700 -1 osd.3 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:00:11.759785+0000 (oldest deadline 2019-12-09T19:00:51.759785+0000) 2019-12-09T19:01:28.028 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:28.024+0000 1694f700 -1 osd.3 344 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:01:28.442 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:01:28.436+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e344 v344) 2019-12-09T19:01:28.612 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:01:28.604+0000 1694f700 -1 osd.4 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:51.664509+0000 front 2019-12-09T18:59:51.682050+0000 (oldest deadline 2019-12-09T19:00:30.486482+0000) 2019-12-09T19:01:28.713 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:01:28.708+0000 1694f700 -1 osd.5 344 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T18:59:50.877249+0000 front 2019-12-09T18:59:44.868655+0000 (oldest deadline 2019-12-09T19:01:15.966864+0000) 2019-12-09T19:01:29.318 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:01:29.076+0000 1694f700 -1 osd.3 345 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e341) 2019-12-09T19:02:03.488 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:02:03.467+0000 3aa81700 -1 osd.6 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:02:03.848 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:02:03.741+0000 35a77700 -1 osd.6 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:02:03.996 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:02:03.825+0000 34a75700 -1 osd.6 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:02:04.109 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:02:04.086+0000 37a7b700 -1 osd.6 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:02:04.110 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:02:04.102+0000 3a280700 -1 osd.6 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:02:38.453 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:19:35.072 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.456 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:19:35.075 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.465 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:19:35.084 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.469 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:19:35.088 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.480 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:19:35.078 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.487 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:19:35.086 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.744 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:19:35.363 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.771 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:19:35.390 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:38.802 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:19:35.401 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:03:02.803 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:02.797+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:03.535 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:03.529+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:03.755 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:03.750+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:03.979 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:03.973+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:04.288 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:04.279+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:04.507 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:04.501+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:04.584 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:04.578+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:04.788 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:04.781+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:04.963 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:04.956+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:05.317 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:05.311+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:05.548 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:05.515+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:05.553 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:05.546+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:05.797 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:05.791+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:05.954 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:05.948+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:06.331 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:06.325+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:06.493 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:06.482+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:06.513 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:06.506+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:06.801 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:06.793+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:06.914 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:06.906+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:07.017 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:07.011+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:07.356 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:07.349+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:07.503 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:07.498+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:07.538 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:07.532+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:07.797 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:07.790+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:07.910 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:07.905+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:08.060 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:08.054+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:08.394 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:08.386+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:08.506 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:08.499+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:08.510 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:08.503+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:08.829 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:08.821+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:08.897 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:08.891+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:09.042 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:09.036+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:09.423 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:09.416+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:09.519 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:09.511+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:09.523 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:09.517+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:09.820 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:09.811+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:09.927 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:03:09.920+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:23.363996+0000 front 2019-12-09T19:02:19.426223+0000 (oldest deadline 2019-12-09T19:03:03.326352+0000) 2019-12-09T19:03:10.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:10.001+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:10.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:10.389+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:10.542 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:10.537+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:10.565 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:10.559+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:10.856 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:10.849+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:11.015 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:11.008+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:11.388 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:11.380+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:11.523 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:11.516+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:11.526 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:11.520+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:11.875 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:11.866+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:11.992 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:11.987+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:12.424 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:12.409+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:12.535 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:12.530+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:12.547 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:12.543+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:12.876 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:12.870+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:13.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:13.038+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:13.402 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:13.396+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:13.495 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:13.490+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:13.506 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:13.501+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:13.870 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:13.863+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:14.091 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:14.084+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:14.406 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:14.396+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:14.513 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:14.508+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:14.535 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:14.530+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:14.902 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:14.895+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:15.077 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:15.072+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:15.447 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:15.439+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:15.476 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:15.471+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:15.584 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:15.579+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:15.891 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:15.884+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:16.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:16.068+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:16.468 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:16.462+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:16.473 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:16.469+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:16.634 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:16.629+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:16.935 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:16.927+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:17.088 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:17.082+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:17.455 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:17.449+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:17.505 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:17.500+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:17.669 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:17.665+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:17.983 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:17.976+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:18.091 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:18.085+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:18.437 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:18.430+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:18.493 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:18.487+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:18.724 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:18.719+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:18.993 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:18.987+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:19.084 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:19.079+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:19.107 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:19.101+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:19.405 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:19.399+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:19.503 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:19.499+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:19.716 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:19.711+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:20.042 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:20.035+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:20.048 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:20.041+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:20.108 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:20.103+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:20.395 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:20.389+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:20.552 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:20.545+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:20.710 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:20.703+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:21.064 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:21.058+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:21.098 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:21.084+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:21.121 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:21.115+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:21.444 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:21.438+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:21.515 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:21.509+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:21.666 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:21.660+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:22.067 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:22.061+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:22.088 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:22.074+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:22.143 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:22.135+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:22.415 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:22.409+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:22.479 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:22.473+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:22.630 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:22.624+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:23.067 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:23.056+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:23.073 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:23.066+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:23.110 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:23.104+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:23.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:23.367+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:23.452 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:03:23.445+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.452714+0000 front 2019-12-09T19:02:22.451520+0000 (oldest deadline 2019-12-09T19:03:03.104412+0000) 2019-12-09T19:03:23.614 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:23.607+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:24.017 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:24.011+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:24.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:24.027+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:24.101 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:24.087+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:24.358 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:24.353+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:24.614 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:24.607+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:25.051 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:25.033+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:25.062 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:25.056+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:25.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:25.063+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:25.389 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:25.382+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:25.599 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:25.592+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:26.037 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:26.031+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:26.038 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:26.032+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:26.047 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:26.042+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:26.406 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:26.398+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:26.648 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:26.642+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:27.003 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:26.996+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:27.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:27.021+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:27.091 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:27.085+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:27.389 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:27.383+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:27.681 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:27.675+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:28.015 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:28.008+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:28.039 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:28.033+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:28.065 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:28.059+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:28.363 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:28.349+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:28.651 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:28.644+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:29.008 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:29.002+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:29.030 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:29.023+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:29.093 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:29.074+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:29.322 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:29.313+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:29.610 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:29.604+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:30.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:30.016+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:30.061 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:30.053+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:30.097 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:30.091+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:30.290 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:30.284+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:30.598 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:30.592+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:31.029 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:31.021+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:31.031 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:31.025+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:31.141 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:31.132+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:31.249 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:31.242+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:31.622 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:31.617+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:32.067 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:32.060+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:32.080 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:32.073+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:32.159 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:32.152+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:32.219 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:32.211+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:32.634 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:32.629+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:33.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:33.070+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:33.111 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:33.105+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:33.129 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:33.123+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:33.194 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:33.186+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:33.642 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:33.636+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:34.086 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:34.080+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:34.111 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:34.100+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:34.133 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:34.125+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:34.157 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:34.150+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:34.677 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:34.670+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:35.040 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:35.034+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:35.113 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:35.107+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:35.171 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:35.164+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:35.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:35.200+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:35.688 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:35.680+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:36.091 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:36.084+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:36.131 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:36.125+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:36.150 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:36.143+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:36.213 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:36.208+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:36.658 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:36.651+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:37.134 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:37.126+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:37.180 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:37.174+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:37.183 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:37.177+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:37.262 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:37.255+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:37.618 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:37.612+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:38.169 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:38.161+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:38.219 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:38.212+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:38.231 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:38.225+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:38.253 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:38.236+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:38.628 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:38.621+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:39.140 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:39.135+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:39.159 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:03:39.154+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e351 v351) 2019-12-09T19:03:39.204 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:39.197+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:39.211 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:39.205+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:39.221 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:39.214+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:39.629 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:39.619+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:40.121 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:40.109+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:40.170 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:40.165+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:40.211 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:40.206+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:40.220 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:40.215+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:40.613 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:40.607+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:41.074 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:41.068+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:41.171 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:41.163+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:41.196 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:41.190+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:41.263 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:41.256+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:41.592 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:41.584+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:42.048 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:42.042+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:42.150 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:42.145+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:42.193 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:42.186+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:42.290 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:42.285+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:42.585 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:42.579+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:43.091 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:43.085+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:43.148 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:43.141+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:43.157 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:43.150+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:43.267 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:43.260+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:43.545 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:43.539+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:03:44.094 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:03:44.086+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.731920+0000 front 2019-12-09T19:02:22.622427+0000 (oldest deadline 2019-12-09T19:03:02.603801+0000) 2019-12-09T19:03:44.116 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:03:44.109+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:20.781352+0000 front 2019-12-09T19:02:21.992138+0000 (oldest deadline 2019-12-09T19:03:04.140069+0000) 2019-12-09T19:03:44.213 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:03:44.203+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:21.686163+0000 front 2019-12-09T19:02:22.450191+0000 (oldest deadline 2019-12-09T19:03:18.610304+0000) 2019-12-09T19:03:44.226 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:03:44.219+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 43sec e351 v351) 2019-12-09T19:03:44.274 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:03:44.267+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:22.528890+0000 front 2019-12-09T19:02:22.777728+0000 (oldest deadline 2019-12-09T19:03:05.968900+0000) 2019-12-09T19:03:44.553 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:03:44.548+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:02:19.426498+0000 front 2019-12-09T19:02:19.678344+0000 (oldest deadline 2019-12-09T19:03:04.091280+0000) 2019-12-09T19:04:14.718 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:21:11.336 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.773 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:21:11.386 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.784 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:21:11.400 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.784 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:21:11.402 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.804 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:21:11.402 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.807 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:21:11.406 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.879 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:21:11.497 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.901 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:21:11.509 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:14.938 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:21:11.536 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:28.145 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:27.867+0000 3827c700 -1 osd.0 355 build_incremental_map_msg missing incremental map 355 2019-12-09T19:04:28.161 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:27.905+0000 124b0700 -1 osd.0 355 build_incremental_map_msg missing incremental map 355 2019-12-09T19:04:28.221 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:27.972+0000 35276700 -1 osd.0 355 build_incremental_map_msg missing incremental map 355 2019-12-09T19:04:28.339 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:28.023+0000 39a7f700 -1 osd.0 355 build_incremental_map_msg missing incremental map 355 2019-12-09T19:04:28.366 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:28.038+0000 3aa81700 -1 osd.0 355 build_incremental_map_msg missing incremental map 355 2019-12-09T19:04:40.274 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:40.268+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:41.326 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:41.318+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:41.334 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:41.331+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:41.721 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:41.715+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:41.933 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:41.928+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:42.293 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:42.286+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:42.326 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:42.321+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:42.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:42.344+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:42.738 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:42.730+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:42.986 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:42.982+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:43.301 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:43.292+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:43.302 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:43.293+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:43.317 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:43.307+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:43.319 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:43.316+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:43.765 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:43.754+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:43.983 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:43.979+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:44.306 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:44.296+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:44.317 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:44.310+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:44.333 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:44.330+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:44.340 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:44.334+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:44.716 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:44.710+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:45.022 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:45.017+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:45.296 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:45.292+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:45.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:45.299+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:45.308 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:45.303+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:45.313 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:45.309+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:45.690 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:45.681+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:46.057 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:46.048+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:46.307 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:46.299+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:46.309 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:46.305+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:46.322 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:46.316+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:46.341 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:46.337+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:46.740 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:46.734+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:47.091 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:47.088+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:47.317 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:47.306+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:47.317 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:47.298+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:47.317 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:47.304+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:47.323 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:47.319+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:47.765 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:47.758+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:48.118 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:48.113+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:48.292 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:48.288+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:48.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:48.299+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:48.323 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:48.319+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:48.339 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:48.333+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:48.722 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:48.715+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:49.126 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:49.121+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:49.253 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:49.248+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:49.302 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:49.297+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:49.347 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:49.341+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:49.363 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:49.356+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:49.726 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:49.720+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:49.952 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:21:46.569 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.001 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:21:46.614 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.023 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:21:46.641 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.037 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:21:46.645 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.073 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:21:46.670 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.074 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:21:46.673 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.085 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:21:46.702 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.096 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:21:46.715 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.167 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:21:46.766 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:50.172 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:50.167+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:50.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:50.270+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:50.311 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:50.307+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:50.389 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:50.383+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:50.393 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:50.387+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:50.746 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:50.741+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:51.138 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:51.133+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:51.290 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:51.285+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:51.334 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:51.329+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:51.385 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:51.368+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:51.386 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:51.381+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:51.773 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:51.766+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:52.106 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:52.100+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:52.270 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:52.265+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:52.293 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:52.288+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:52.359 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:52.353+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:52.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:52.362+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:52.786 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:52.780+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:53.129 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:53.123+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:53.280 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:53.267+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:53.280 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:53.267+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:53.339 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:53.332+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:53.387 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:53.380+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:53.773 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:53.766+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:54.161 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:54.152+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:54.280 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:54.275+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:54.299 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:54.293+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:54.327 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:54.316+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:54.419 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:54.413+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:54.771 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:54.759+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:55.156 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:55.150+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:55.336 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:55.330+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:55.340 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:55.335+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:55.375 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:55.356+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:55.452 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:55.446+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:55.774 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:55.768+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:56.144 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:56.139+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:56.308 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:56.300+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:56.322 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:56.317+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:56.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:56.345+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:56.498 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:56.492+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:56.804 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:56.798+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:57.172 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:57.167+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:57.319 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:57.313+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:57.320 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:57.314+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:57.338 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:57.332+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:57.523 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:57.517+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:57.822 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:57.816+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:58.214 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:58.206+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:58.299 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:58.294+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:58.336 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:58.330+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:58.367 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:58.361+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:58.523 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:58.517+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:58.830 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:58.820+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:59.246 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:04:59.240+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:04:59.333 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:04:59.324+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:04:59.343 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:04:59.336+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:04:59.357 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:04:59.349+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:04:59.506 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:04:59.499+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:04:59.859 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:04:59.851+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:04:59.879 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:04:59.873+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:00.249 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:00.243+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:00.333 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:00.327+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:00.345 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:00.339+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:00.363 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:00.357+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:00.465 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:00.454+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:00.858 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:00.852+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:00.863 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:00.856+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:01.271 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:01.266+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:01.351 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:01.346+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:01.389 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:01.382+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:01.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:01.410+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:01.429 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:01.424+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:01.492 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T19:05:01.467+0000 20e25700 -1 RGWWatcher::handle_error cookie 5464378272 err (107) Transport endpoint is not connected 2019-12-09T19:05:01.886 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:01.880+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:01.890 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:01.885+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:01.904 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:01.897+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:02.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:02.234+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:02.327 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:02.321+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:02.411 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:02.406+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:02.439 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:02.432+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:02.440 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:02.433+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:02.906 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:02.898+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:02.938 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:02.932+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:02.939 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:02.933+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:03.286 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:03.279+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:03.313 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:03.308+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:03.396 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:03.391+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:03.404 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:03.398+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:03.470 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:03.464+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:03.941 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:03.934+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:03.960 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:03.953+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:03.961 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:03.954+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:04.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:04.309+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:04.337 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:04.330+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:04.415 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:04.409+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:04.432 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:04.420+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:04.492 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:04.486+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:04.982 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:04.975+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:05.014 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:05.004+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:05.015 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:05.006+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:05.291 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:05.286+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:05.354 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:05.348+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:05.415 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:05.407+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:05.422 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:05.416+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:05.494 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:05.488+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:05.974 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:05.968+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:06.055 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:06.048+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:06.064 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:06.053+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:06.289 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:06.283+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:06.377 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:06.367+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:06.387 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:06.380+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:06.387 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:06.383+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:06.398 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:06.391+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:06.490 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:06.485+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:06.966 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:06.960+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:07.032 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:07.025+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:07.033 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:07.027+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:07.256 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:07.249+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:07.376 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:07.367+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:07.382 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:07.370+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:07.385 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:07.379+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:07.424 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:07.417+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:07.489 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:07.481+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:08.007 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:08.001+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:08.044 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:08.037+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:08.045 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:08.039+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:08.262 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:08.257+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:08.352 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:08.347+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:08.356 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:08.350+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:08.381 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:08.374+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:08.399 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:08.394+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:08.475 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:08.468+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:08.964 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:08.957+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:09.037 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:09.030+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:09.037 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:09.032+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:09.263 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:09.257+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:09.387 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:09.378+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:09.394 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:09.388+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:09.394 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:09.389+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:09.416 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:09.409+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:09.515 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:09.509+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:09.933 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:09.927+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:10.067 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:10.059+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:10.067 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:10.061+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:10.302 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:10.298+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:10.359 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:10.354+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:10.361 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:10.358+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:10.410 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:10.405+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:10.465 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:10.459+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:10.568 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:10.562+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:10.925 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:10.919+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:11.031 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:11.026+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:11.035 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:11.031+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:11.334 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:11.329+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:11.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:11.342+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:11.349 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:11.344+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:11.369 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:11.363+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:11.443 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:11.439+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:11.590 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:11.584+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:11.985 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:11.975+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:11.993 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:11.988+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:11.996 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:11.991+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:12.303 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:12.294+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:12.384 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:12.379+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:12.386 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:12.380+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:12.400 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:12.396+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:12.464 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:12.459+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:12.577 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:12.571+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:12.971 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:12.960+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:12.979 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:12.974+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:12.982 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:12.978+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:13.271 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:13.266+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:13.347 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:13.341+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:13.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:13.343+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:13.366 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:13.361+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:13.513 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:13.508+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:13.547 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:13.543+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:13.961 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:13.955+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:13.962 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:13.957+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:14.023 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:14.017+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:14.326 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:14.319+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:14.327 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:14.322+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:14.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:14.390+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:14.396 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:14.392+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:14.543 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:14.539+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:14.569 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:14.564+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:14.925 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:14.919+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:14.927 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:14.922+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:15.022 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:05:15.017+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e356 v356) 2019-12-09T19:05:15.027 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:15.022+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:15.295 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:15.289+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:15.322 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:15.310+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:15.365 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:15.358+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:15.365 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:15.359+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:15.510 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:15.504+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:15.534 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:15.528+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:15.910 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:15.904+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:15.910 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:15.906+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:16.002 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:15.997+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:16.320 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:16.315+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:16.372 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:16.362+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:16.378 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:16.372+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:16.379 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:16.374+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:16.544 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:16.539+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:16.558 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:16.552+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:16.872 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:16.867+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:16.875 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:16.870+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:17.052 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:17.047+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:17.298 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:17.289+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:17.377 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:17.369+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:17.381 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:17.377+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:17.382 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:17.378+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:17.522 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:17.517+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:17.531 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:17.527+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:17.944 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:17.927+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:17.947 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:17.932+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:18.075 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:18.069+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:18.315 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:18.310+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:18.386 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:18.375+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:18.428 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:18.423+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:18.431 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:18.426+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:18.525 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:18.521+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:18.558 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:18.554+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:18.955 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:18.948+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:18.956 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:18.950+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:19.062 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:19.057+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:19.365 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:19.358+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:19.405 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:19.399+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:19.453 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:19.448+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:19.457 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:19.452+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:19.502 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:19.497+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:19.586 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:19.581+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:19.970 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:19.966+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:19.974 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:19.970+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:20.034 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:20.024+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:20.050 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:05:20.039+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e356 v356) 2019-12-09T19:05:20.370 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:20.364+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:20.444 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:20.439+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:20.473 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:20.465+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:20.473 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:20.467+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:20.473 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:20.465+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:20.596 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:20.591+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:20.969 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:20.963+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:20.970 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:20.965+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:21.058 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:21.052+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:21.374 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:21.368+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:21.423 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:21.419+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:21.444 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:21.438+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:21.514 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:21.507+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:21.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:21.509+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:21.571 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:21.565+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:21.972 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:21.966+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:21.974 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:21.968+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:22.050 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:22.044+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:22.405 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:22.399+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:22.439 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:22.434+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:22.493 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:22.488+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:22.496 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:22.491+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:22.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:22.492+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:22.594 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:22.589+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:23.002 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:22.995+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:23.002 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:22.997+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:23.033 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:23.027+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:23.370 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:23.365+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:23.454 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:23.449+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:23.458 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:23.453+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:23.538 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:23.531+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:23.539 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:23.533+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:23.569 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:23.555+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:24.043 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:24.036+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:24.044 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:24.039+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:24.073 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:24.067+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:24.341 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:24.336+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:24.413 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:24.408+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:24.486 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:24.480+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:24.529 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:24.524+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:24.532 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:24.528+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:24.557 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:24.551+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:25.004 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:24.999+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:25.008 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:25.003+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:25.057 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:25.051+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:25.064 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:05:25.059+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e356 v356) 2019-12-09T19:05:25.335 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:25.330+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:25.376 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:25.370+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:25.539 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:25.532+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:25.552 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:25.547+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:25.569 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:25.563+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:25.574 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:25.565+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:26.042 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:26.036+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:26.042 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:26.038+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:26.091 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:26.086+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:26.331 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:26.325+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:26.366 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:26.362+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:26.528 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:26.523+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:26.528 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:26.524+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:26.582 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:26.572+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:26.583 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:26.579+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:27.023 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:27.016+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:27.023 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:27.018+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:27.071 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:27.065+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:27.364 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:27.360+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:27.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:27.394+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:27.573 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:27.567+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:27.574 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:27.569+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:27.597 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:27.592+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:27.613 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:27.609+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:28.012 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:28.005+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:28.012 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:28.007+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:28.053 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:28.046+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:28.361 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:28.356+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:28.386 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:28.381+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:28.582 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:28.577+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:28.583 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:28.579+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:28.584 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:28.579+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:28.666 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:28.662+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:28.989 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:28.983+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:28.990 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:28.985+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:29.016 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:29.010+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:29.324 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:29.319+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:29.373 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:29.368+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:29.571 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:29.566+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:29.590 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:29.585+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:29.590 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:29.587+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:29.659 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:29.653+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:30.005 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:29.994+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:30.006 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:29.997+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:30.061 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:30.054+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:30.088 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:05:30.080+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e356 v356) 2019-12-09T19:05:30.364 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:30.360+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:30.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:30.369+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:30.592 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:30.586+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:30.621 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:30.616+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:30.624 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:30.618+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:30.624 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:30.621+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:30.973 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:30.967+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:30.975 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:30.969+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:31.020 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:31.014+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:31.369 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:31.359+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:31.395 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:31.389+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:31.585 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:31.580+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:31.608 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:31.604+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:31.677 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:31.669+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:31.677 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:31.672+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:31.951 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:31.944+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:31.952 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:31.946+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:32.058 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:32.053+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:32.363 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:32.359+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:32.373 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:32.369+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:32.550 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:32.546+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:32.615 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:32.609+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:32.721 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:32.711+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:32.721 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:32.715+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:32.971 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:32.965+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:32.972 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:32.967+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:33.061 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:33.056+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:33.343 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:33.337+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:33.384 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:33.379+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:33.588 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:33.584+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:33.600 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:33.595+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:33.719 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:33.713+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:33.719 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:33.715+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:33.933 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:33.927+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:33.934 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:33.929+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:34.088 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:34.083+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:34.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:34.341+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:34.349 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:34.344+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:34.596 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:34.589+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:34.611 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:34.607+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:34.698 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:34.692+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:34.698 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:34.694+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:34.890 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:34.884+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:34.891 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:34.886+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:35.106 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:35.101+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:35.141 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:05:35.108+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e356 v356) 2019-12-09T19:05:35.344 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:35.338+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:35.375 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:35.371+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:35.613 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:35.607+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:35.666 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:35.662+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:35.675 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:35.670+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:35.678 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:35.675+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:35.924 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:35.917+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:35.924 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:35.920+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:36.069 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:36.064+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:36.328 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:36.321+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:36.357 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:36.351+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:36.635 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:36.630+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:36.651 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:36.646+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:36.652 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:36.647+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:36.655 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:36.651+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:36.929 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:36.924+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:36.933 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:36.928+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:37.093 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:37.089+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:37.351 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:37.346+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:37.382 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:37.378+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:37.603 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:37.597+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:37.627 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:37.622+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:37.650 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:37.644+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:37.653 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:37.646+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:37.977 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:37.970+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:37.978 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:37.972+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:38.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:38.098+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:38.362 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:38.357+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:38.362 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:38.357+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:38.577 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:38.572+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:38.622 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:38.615+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:38.624 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:38.618+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:38.652 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:38.648+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:38.952 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:38.946+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:38.953 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:38.948+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:39.144 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:39.138+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:39.323 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:39.317+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:39.405 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:39.399+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:39.595 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:39.590+0000 1694f700 -1 osd.4 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:54.759852+0000 front 2019-12-09T19:03:54.449338+0000 (oldest deadline 2019-12-09T19:04:40.336741+0000) 2019-12-09T19:05:39.598 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:39.594+0000 1694f700 -1 osd.4 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:39.630 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:05:39.625+0000 1694f700 -1 osd.7 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.127883+0000 front 2019-12-09T19:04:00.729865+0000 (oldest deadline 2019-12-09T19:04:42.003329+0000) 2019-12-09T19:05:39.674 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:05:39.668+0000 1694f700 -1 osd.1 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:03:56.220195+0000 front 2019-12-09T19:03:56.399509+0000 (oldest deadline 2019-12-09T19:04:40.110163+0000) 2019-12-09T19:05:39.978 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:39.971+0000 1694f700 -1 osd.3 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:04:21.562256+0000 (oldest deadline 2019-12-09T19:05:01.562256+0000) 2019-12-09T19:05:39.982 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:39.975+0000 1694f700 -1 osd.3 356 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:40.104 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:05:40.096+0000 1694f700 -1 osd.2 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:00.518445+0000 front 2019-12-09T19:04:02.042771+0000 (oldest deadline 2019-12-09T19:04:40.717739+0000) 2019-12-09T19:05:40.196 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:05:40.186+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e356 v356) 2019-12-09T19:05:40.373 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:05:40.368+0000 1694f700 -1 osd.0 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:02.277184+0000 front 2019-12-09T19:04:00.244384+0000 (oldest deadline 2019-12-09T19:04:42.422022+0000) 2019-12-09T19:05:40.390 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:05:40.386+0000 1694f700 -1 osd.5 356 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:04:01.742069+0000 front 2019-12-09T19:04:02.243783+0000 (oldest deadline 2019-12-09T19:04:41.663501+0000) 2019-12-09T19:05:40.671 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:40.646+0000 1694f700 -1 osd.4 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:41.473 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:05:41.191+0000 1694f700 -1 osd.3 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e355) 2019-12-09T19:05:41.664 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:41.659+0000 1694f700 -1 osd.4 357 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:05:42.910 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:05:42.885+0000 1694f700 -1 osd.4 359 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:126264 5.3 5:c609908c:::notify.5:head [watch ping cookie 5464378272 gen 3] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-09T19:06:31.484 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:23:28.102 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.488 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:23:28.107 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.497 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:23:28.116 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.500 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:23:28.119 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.502 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:23:28.101 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.514 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:23:28.112 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.635 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:23:28.254 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.673 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:23:28.292 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:31.712 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:23:28.311 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:28.412 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:28.404+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:28.449 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:28.442+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:28.969 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:28.961+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:29.217 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:29.211+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:29.398 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:29.391+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:29.408 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:29.400+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:29.433 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:29.427+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:30.011 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:30.004+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:30.188 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:30.181+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:30.424 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:30.419+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:30.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:30.419+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:30.426 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:30.422+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:30.428 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:30.423+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:30.609 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:30.603+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:31.062 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:31.055+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:31.184 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:31.179+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:31.383 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:31.377+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:31.392 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:31.387+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:31.396 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:31.390+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:31.434 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:31.428+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:31.641 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:31.634+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:32.072 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:32.064+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:32.157 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:32.150+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:32.356 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:32.350+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:32.407 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:32.401+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:32.441 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:32.435+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:32.465 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:32.459+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:32.691 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:32.683+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:33.066 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:33.059+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:33.171 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:33.166+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:33.320 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:33.314+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:33.414 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:33.407+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:33.455 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:33.448+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:33.458 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:33.452+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:33.681 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:33.675+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:34.076 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:34.069+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:34.194 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:34.188+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:34.315 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:34.309+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:34.432 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:34.425+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:34.466 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:34.457+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:34.470 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:34.463+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:34.695 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:34.689+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:35.094 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:35.087+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:35.232 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:35.225+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:35.370 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:35.363+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:35.404 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:35.398+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:35.441 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:35.435+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:35.517 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:35.510+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:35.737 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:35.731+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:36.057 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:36.049+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:36.196 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:36.190+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:36.362 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:36.355+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:36.418 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:36.412+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:36.454 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:36.448+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:36.544 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:36.537+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:36.691 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:36.684+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:37.069 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:37.062+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:37.197 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:37.191+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:37.336 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:37.330+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:37.388 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:37.381+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:37.446 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:37.439+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:37.525 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:37.517+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:37.716 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:37.704+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:38.030 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:38.023+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:38.165 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:38.159+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:38.344 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:38.337+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:38.393 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:38.386+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:38.408 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:38.402+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:38.527 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:38.520+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:38.691 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:38.684+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:38.997 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:38.991+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:39.217 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:39.211+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:39.326 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:39.321+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:39.371 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:39.366+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:39.435 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:39.428+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:39.559 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:39.551+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:39.733 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:39.726+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:39.981 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:39.973+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:40.176 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:40.169+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:40.284 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:40.279+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:40.354 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:40.348+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:40.396 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:40.390+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:40.513 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:40.507+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:40.729 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:40.723+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:41.030 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:41.024+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:41.131 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:41.126+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:41.241 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:41.235+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:41.360 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:41.353+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:41.402 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:41.396+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:41.507 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:41.501+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:41.698 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:41.684+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:42.022 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:42.014+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:42.106 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:42.100+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:42.248 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:42.242+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:42.368 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:42.362+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:42.429 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:42.423+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:42.501 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:42.495+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:42.707 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:42.700+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:43.070 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:43.062+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:43.110 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:43.105+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:43.239 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:43.234+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:43.350 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:43.340+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:43.425 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:43.418+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:43.538 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:43.531+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:43.708 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:43.699+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:44.110 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:44.103+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:44.139 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:44.133+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:44.284 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:44.278+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:44.337 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:44.332+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:44.470 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:44.466+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:44.534 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:44.518+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:44.697 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:44.692+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:45.097 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:45.090+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:45.118 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:45.113+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:45.320 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:45.316+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:45.362 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:45.358+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:45.453 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:45.447+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:45.567 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:45.560+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:45.706 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:45.700+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:46.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:46.071+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:46.171 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:46.166+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:46.348 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:46.344+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:46.387 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:46.383+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:46.424 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:46.419+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:46.547 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:46.535+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:46.738 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:46.732+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:47.070 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:47.064+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:47.132 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:47.125+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:47.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:47.369+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:47.419 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:47.414+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:47.444 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:47.438+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:47.608 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:47.599+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:47.718 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:47.703+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:48.051 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:48.044+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:48.188 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:48.182+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:48.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:48.370+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:48.436 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:48.431+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:48.467 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:48.462+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:48.629 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:48.624+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:48.724 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:48.718+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:49.032 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:49.022+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:49.220 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:49.216+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:49.374 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:49.366+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:49.482 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:49.478+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:49.522 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:49.516+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:49.585 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:49.579+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:49.706 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:49.700+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:50.019 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:50.014+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:50.205 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:50.201+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:50.423 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:50.417+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:50.460 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:50.454+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:50.534 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:50.528+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:50.584 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:50.579+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:50.716 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:50.709+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:51.034 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:51.021+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:51.172 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:51.167+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:51.404 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:51.398+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:51.443 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:51.438+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:51.540 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:51.534+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:51.599 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:51.594+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:51.684 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:51.678+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:52.008 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:52.002+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:52.187 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:52.182+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:52.421 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:52.412+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:52.468 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:52.463+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:52.575 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:52.571+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:52.632 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:52.626+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:52.664 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:52.658+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:52.968 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:52.960+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:53.213 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:53.198+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:53.439 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:53.432+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:53.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:53.463+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:53.543 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:53.537+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:53.609 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:53.603+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:53.665 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:53.659+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:54.002 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:53.996+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:54.243 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:54.237+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:54.431 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:54.424+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:54.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:54.475+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:54.590 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:54.584+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:54.611 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:54.599+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:54.692 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:54.687+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:55.064 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:55.056+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:55.215 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:55.208+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:55.409 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:55.404+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:55.453 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:55.446+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:55.605 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:55.598+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:55.612 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:55.606+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:55.661 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:55.654+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:56.054 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:56.047+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:56.195 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:56.189+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:56.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:56.417+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:56.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:56.488+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:56.602 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:56.595+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:56.624 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:56.619+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:56.696 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:56.690+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:57.053 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:57.045+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:57.253 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:57.238+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:57.399 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:57.392+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:57.509 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:57.503+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:57.586 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:57.572+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:57.608 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:57.603+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:57.687 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:57.674+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:58.067 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:58.061+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:58.244 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:58.238+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:58.376 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:58.369+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:58.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:58.487+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:58.584 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:58.578+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:58.655 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:58.650+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:58.691 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:58.685+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:07:59.090 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:07:59.085+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:07:59.293 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:07:59.287+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:07:59.415 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:07:59.407+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:07:59.532 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:07:59.527+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:07:59.622 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:07:59.617+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:07:59.657 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:07:59.651+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:07:59.692 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:07:59.686+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:00.072 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:00.065+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:00.338 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:00.333+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:00.465 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:00.459+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:00.537 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:00.530+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:00.585 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:00.579+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:00.652 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:00.646+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:00.657 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:00.650+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:00.677 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:08:00.671+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 40sec e362 v362) 2019-12-09T19:08:01.098 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:01.090+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:01.324 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:01.318+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:01.457 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:01.449+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:01.521 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:01.515+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:01.593 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:01.587+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:01.663 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:01.656+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:01.678 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:01.672+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:02.118 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:02.112+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:02.309 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:02.303+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:02.459 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:02.453+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:02.549 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:02.534+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:02.567 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:02.560+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:02.679 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:02.671+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:02.693 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:02.689+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:03.106 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:03.100+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:03.300 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:03.294+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:03.471 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:03.465+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:03.506 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:03.500+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:03.617 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:03.611+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:03.693 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:03.689+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:03.703 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:03.698+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:04.136 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:04.130+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:04.274 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:04.268+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:04.502 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:04.496+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:04.524 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:04.517+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:04.658 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:04.651+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:04.717 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:04.712+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:04.744 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:04.737+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:05.111 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:05.106+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:05.265 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:05.259+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:05.506 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:05.502+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:05.556 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:05.550+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:05.671 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:05.665+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:05.685 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:05.679+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:05.691 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:08:05.687+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 40sec e362 v362) 2019-12-09T19:08:05.730 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:05.725+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:06.123 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:06.118+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:06.304 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:06.299+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:06.555 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:06.549+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:06.563 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:06.557+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:06.639 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:06.634+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:06.645 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:06.641+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:06.689 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:06.684+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:07.137 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:07.126+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:07.357 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:07.351+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:07.562 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:07.544+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:07.584 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:07.577+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:07.616 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:07.605+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:07.679 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:07.673+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:07.723 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:07.716+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:08.179 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:08.174+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:08.319 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:08.313+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:08.548 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:08.542+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:08.578 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:08.571+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:08.635 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:08.629+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:08.703 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:08.696+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:08.713 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:08.707+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:09.221 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:09.215+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:09.307 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:09.300+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:09.584 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:09.577+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:09.596 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:09.589+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:09.624 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:09.618+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:09.714 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:09.708+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:09.753 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:09.747+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:10.199 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:08:10.193+0000 1694f700 -1 osd.1 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.285303+0000 front 2019-12-09T19:06:47.655004+0000 (oldest deadline 2019-12-09T19:07:28.207600+0000) 2019-12-09T19:08:10.265 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:08:10.259+0000 1694f700 -1 osd.6 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.600106+0000 front 2019-12-09T19:06:43.487672+0000 (oldest deadline 2019-12-09T19:07:28.780168+0000) 2019-12-09T19:08:10.588 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:08:10.582+0000 1694f700 -1 osd.4 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:44.344027+0000 front 2019-12-09T19:06:46.629816+0000 (oldest deadline 2019-12-09T19:07:28.540410+0000) 2019-12-09T19:08:10.592 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:08:10.587+0000 1694f700 -1 osd.5 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.902518+0000 front 2019-12-09T19:06:48.241592+0000 (oldest deadline 2019-12-09T19:07:29.581986+0000) 2019-12-09T19:08:10.626 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:08:10.620+0000 1694f700 -1 osd.2 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:47.840294+0000 front 2019-12-09T19:06:47.846418+0000 (oldest deadline 2019-12-09T19:07:28.318073+0000) 2019-12-09T19:08:10.708 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:08:10.703+0000 1694f700 -1 osd.7 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:48.275269+0000 front 2019-12-09T19:06:47.176045+0000 (oldest deadline 2019-12-09T19:07:28.174847+0000) 2019-12-09T19:08:10.753 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:08:10.746+0000 1694f700 -1 osd.0 362 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:06:46.628641+0000 front 2019-12-09T19:06:46.644015+0000 (oldest deadline 2019-12-09T19:07:29.972746+0000) 2019-12-09T19:08:10.774 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:08:10.764+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 40sec e362 v362) 2019-12-09T19:08:53.022 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:08:52.980+0000 35a77700 -1 osd.3 368 build_incremental_map_msg missing incremental map 368 2019-12-09T19:08:53.950 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:08:53.683+0000 38a7d700 -1 osd.3 369 build_incremental_map_msg missing incremental map 368 2019-12-09T19:08:54.306 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:08:54.170+0000 3aa81700 -1 osd.3 369 build_incremental_map_msg missing incremental map 368 2019-12-09T19:08:55.190 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:08:55.176+0000 3b282700 -1 osd.3 369 build_incremental_map_msg missing incremental map 368 2019-12-09T19:09:09.927 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:09.922+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:10.102 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:10.094+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:10.222 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:10.217+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:10.544 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:10.539+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:10.903 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:10.897+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:11.129 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:11.124+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:11.257 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:11.247+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:11.533 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:11.523+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:11.858 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:11.852+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:12.097 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:12.091+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:12.131 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:12.125+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:12.304 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:12.298+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:12.535 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:12.530+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:12.889 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:12.883+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:13.124 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:13.116+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:13.181 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:13.174+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:13.181 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:13.173+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:13.354 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:13.349+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:13.525 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:13.519+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:13.912 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:13.905+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:14.150 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:14.145+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:14.194 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:14.190+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:14.208 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:14.203+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:14.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:14.361+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:14.547 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:14.541+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:14.887 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:14.883+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:15.170 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:15.165+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:15.171 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:15.167+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:15.241 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:15.236+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:15.368 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:15.363+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:15.565 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:15.558+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:15.881 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:15.875+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:16.128 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:16.124+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:16.187 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:16.183+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:16.262 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:16.255+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:16.366 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:16.361+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:16.554 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:16.548+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:16.911 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:16.906+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:17.143 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:17.139+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:17.151 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:17.146+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:17.271 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:17.267+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:17.377 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:17.371+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:17.584 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:17.578+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:17.922 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:17.918+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:18.128 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:18.124+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:18.184 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:18.179+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:18.309 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:18.303+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:18.425 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:18.418+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:18.548 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:18.542+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:18.882 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:18.875+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:19.149 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:19.144+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:19.214 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:19.210+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:19.315 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:19.309+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:19.445 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:19.439+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:19.581 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:19.576+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:19.840 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:19.835+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:20.169 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:20.164+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:20.207 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:20.202+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:20.350 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:20.345+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:20.425 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:20.420+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:20.536 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:20.531+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:20.828 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:20.824+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:21.139 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:21.134+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:21.210 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:21.207+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:21.356 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:21.352+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:21.438 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:21.434+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:21.469 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:21.464+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:21.531 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:21.525+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:21.821 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:21.818+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:22.099 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:22.095+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:22.227 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:22.222+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:22.317 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:22.312+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:22.452 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:22.447+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:22.513 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:22.507+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:22.540 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:22.535+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:22.783 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:22.778+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:23.133 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:23.129+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:23.219 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:23.215+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:23.324 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:23.319+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:23.469 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:23.465+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:23.520 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:23.514+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:23.547 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:23.542+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:23.768 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:23.763+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:24.156 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:24.151+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:24.200 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:24.195+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:24.302 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:24.296+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:24.490 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:24.483+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:24.526 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:24.522+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:24.578 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:24.574+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:24.816 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:24.810+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:25.136 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:25.131+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:25.209 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:25.204+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:25.315 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:25.310+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:25.507 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:25.502+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:25.548 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:25.543+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:25.623 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:25.618+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:25.853 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:25.849+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:26.144 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:26.139+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:26.253 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:26.245+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:26.281 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:26.276+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:26.528 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:26.524+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:26.581 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:26.575+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:26.643 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:26.638+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:26.828 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:26.822+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:27.117 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:27.112+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:27.226 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:27.221+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:27.308 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:27.302+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:27.573 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:27.567+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:27.574 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:27.569+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:27.634 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:27.629+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:27.661 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:27.653+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:27.785 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:27.780+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:28.169 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:28.164+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:28.277 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:28.272+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:28.345 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:28.338+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:28.631 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:28.626+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:28.638 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:28.633+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:28.652 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:28.647+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:28.677 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:28.673+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:28.754 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:28.749+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:29.157 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:29.151+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:29.263 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:29.258+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:29.331 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:29.325+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:29.595 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:29.590+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:29.596 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:29.591+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:29.673 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:29.668+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:29.750 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:29.726+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:29.761 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:29.756+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:30.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:30.129+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:30.280 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:30.275+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:30.334 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:30.330+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:30.632 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:30.626+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:30.633 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:30.628+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:30.710 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:30.701+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:30.774 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:30.769+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:30.790 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:30.785+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:31.137 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:31.132+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:31.317 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:31.311+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:31.377 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:31.372+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:31.646 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:31.638+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:31.647 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:31.640+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:31.672 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:31.667+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:31.757 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:31.752+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:31.778 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:31.773+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:32.173 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:32.167+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:32.362 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:32.357+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:32.365 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:32.359+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:32.646 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:32.640+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:32.694 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:32.688+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:32.697 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:32.692+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:32.759 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:32.754+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:32.828 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:32.823+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:33.216 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:33.210+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:33.365 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:33.357+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:33.414 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:33.410+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:33.683 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:33.676+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:33.695 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:33.690+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:33.698 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:33.693+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:33.816 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:33.811+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:33.856 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:33.851+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:34.209 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:34.204+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:34.369 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:34.364+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:34.462 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:34.456+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:34.660 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:34.655+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:34.661 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:34.656+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:34.688 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:34.684+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:34.844 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:34.839+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:34.892 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:34.887+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:35.164 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:35.159+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:35.230 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T19:09:35.217+0000 20e25700 -1 RGWWatcher::handle_error cookie 5402667728 err (107) Transport endpoint is not connected 2019-12-09T19:09:35.390 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:35.384+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:35.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:35.491+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:35.682 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:35.676+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:35.683 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:35.678+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:35.746 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:35.736+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:35.823 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:35.818+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:35.889 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:35.884+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:36.184 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:36.179+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:36.382 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:36.377+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:36.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:36.474+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:36.696 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:36.690+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:36.696 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:36.691+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:36.699 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:36.694+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:36.833 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:36.825+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:36.941 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:36.932+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:37.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:37.235+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:37.410 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:37.404+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:37.443 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:37.436+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:37.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:37.682+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:37.728 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:37.717+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:37.728 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:37.718+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:37.876 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:37.870+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:37.917 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:37.906+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:38.240 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:38.234+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:38.407 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:38.401+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:38.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:38.473+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:38.664 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:38.659+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:38.766 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:38.760+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:38.767 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:38.762+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:38.882 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:38.877+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:38.900 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:38.894+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:39.209 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:39.203+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:39.411 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:39.403+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:39.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:39.478+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:39.490 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:39.481+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:39.659 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:39.652+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:39.754 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:39.745+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:39.758 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:39.747+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:39.866 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:39.860+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:39.912 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:39.907+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:40.205 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:40.199+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:40.393 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:40.387+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:40.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:40.489+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:40.495 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:40.491+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:40.652 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:40.646+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:40.717 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:40.707+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:40.717 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:40.708+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:40.896 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:40.891+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:40.917 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:40.912+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:41.084 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:09:41.077+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e369 v369) 2019-12-09T19:09:41.189 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:41.184+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:41.396 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:41.390+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:41.456 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:41.452+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:41.458 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:41.454+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:41.685 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:41.678+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:41.729 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:41.721+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:41.729 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:41.723+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:41.935 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:41.929+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:41.935 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:41.930+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:42.217 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:42.212+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:42.349 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:42.344+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:42.467 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:42.462+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:42.467 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:42.463+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:42.724 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:42.719+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:42.727 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:42.722+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:42.729 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:42.724+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:42.915 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:42.910+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:42.965 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:42.961+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:43.266 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:43.260+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:43.350 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:43.343+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:43.522 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:43.516+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:43.522 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:43.517+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:43.729 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:43.721+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:43.730 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:43.723+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:43.765 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:43.758+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:43.952 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:43.945+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:43.967 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:43.961+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:44.272 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:44.266+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:44.333 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:44.324+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:44.545 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:44.541+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:44.547 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:44.543+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:44.738 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:44.732+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:44.739 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:44.734+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:44.750 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:44.745+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:44.911 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:44.905+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:45.018 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:45.013+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:45.255 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:45.250+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:45.317 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:45.303+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:45.531 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:45.525+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:45.532 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:45.526+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:45.754 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:45.747+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:45.755 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:45.749+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:45.783 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:45.777+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:45.880 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:45.874+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:46.070 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:46.065+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:46.099 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:09:46.093+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e369 v369) 2019-12-09T19:09:46.281 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:46.276+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:46.358 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:46.353+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:46.559 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:46.553+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:46.559 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:46.554+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:46.723 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:46.715+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:46.724 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:46.717+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:46.811 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:46.806+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:46.853 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:46.847+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:47.079 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:47.074+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:47.305 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:47.297+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:47.400 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:47.394+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:47.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:47.594+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:47.600 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:47.595+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:47.743 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:47.735+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:47.743 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:47.737+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:47.772 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:47.766+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:47.830 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:47.819+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:48.042 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:48.037+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:48.261 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:48.252+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:48.452 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:48.446+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:48.625 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:48.619+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:48.626 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:48.621+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:48.787 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:48.779+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:48.788 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:48.782+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:48.817 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:48.810+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:48.841 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:48.835+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:49.083 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:49.074+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:49.310 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:49.304+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:49.427 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:49.421+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:49.614 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:49.608+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:49.615 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:49.610+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:49.781 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:49.774+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:49.782 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:49.777+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:49.807 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:49.800+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:49.847 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:49.840+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:50.058 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:50.053+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:50.329 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:50.324+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:50.440 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:50.429+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:50.597 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:50.592+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:50.598 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:50.593+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:50.769 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:50.764+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:50.772 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:50.767+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:50.844 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:50.838+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:50.853 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:50.846+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:51.026 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:51.021+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:51.115 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:09:51.110+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e369 v369) 2019-12-09T19:09:51.386 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:51.381+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:51.431 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:51.425+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:51.617 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:51.610+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:51.617 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:51.612+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:51.802 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:51.795+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:51.803 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:51.797+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:51.826 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:51.820+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:51.885 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:51.878+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:51.983 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:51.978+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:52.377 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:52.372+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:52.394 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:52.387+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:52.667 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:52.663+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:52.669 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:52.666+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:52.834 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:52.828+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:52.855 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:52.849+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:52.858 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:52.853+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:52.885 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:52.880+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:52.952 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:52.947+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:53.404 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:53.397+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:53.407 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:53.401+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:53.685 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:53.678+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:53.685 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:53.680+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:53.873 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:53.866+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:53.874 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:53.868+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:53.879 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:53.872+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:53.908 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:53.901+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:53.943 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:53.937+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:54.437 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:54.431+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:54.453 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:54.447+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:54.672 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:54.665+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:54.673 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:54.668+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:54.874 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:54.867+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:54.875 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:54.868+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:54.924 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:54.918+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:54.927 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:54.921+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:54.978 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:54.971+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:55.407 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:55.402+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:55.485 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:55.478+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:55.630 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:55.624+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:55.631 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:55.626+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:55.901 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:55.889+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:55.901 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:55.891+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:55.936 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:55.917+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:55.946 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:55.942+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:55.968 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:55.962+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:56.133 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:09:56.127+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e369 v369) 2019-12-09T19:09:56.460 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:56.454+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:56.518 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:56.512+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:56.613 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:56.607+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:56.616 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:56.610+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:56.862 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:56.855+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:56.862 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:56.857+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:56.897 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:56.891+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:56.946 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:56.941+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:56.972 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:56.967+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:57.498 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:57.493+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:57.527 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:57.521+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:57.607 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:57.600+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:57.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:57.602+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:57.851 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:57.845+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:57.851 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:57.846+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:57.909 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:57.904+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:57.930 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:57.924+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:57.940 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:57.935+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:58.475 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:58.469+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:58.565 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:58.559+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:58.609 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:58.602+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:58.609 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:58.604+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:58.833 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:58.827+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:58.835 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:58.830+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:58.916 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:58.910+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:09:58.940 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:58.935+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:58.952 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:58.946+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:59.507 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:09:59.498+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:09:59.533 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:09:59.527+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:09:59.605 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:59.600+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:09:59.606 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:09:59.602+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:09:59.792 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:59.786+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:09:59.796 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:09:59.791+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:09:59.912 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:09:59.905+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:09:59.946 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:09:59.942+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:09:59.951 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:09:59.944+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:10:00.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:10:00.544+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:10:00.551 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:10:00.546+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:28.443394+0000 front 2019-12-09T19:08:28.450257+0000 (oldest deadline 2019-12-09T19:09:12.218561+0000) 2019-12-09T19:10:00.635 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:10:00.629+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:10:00.635 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:10:00.631+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:10:00.812 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:10:00.805+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:08:47.279635+0000 (oldest deadline 2019-12-09T19:09:27.279635+0000) 2019-12-09T19:10:00.812 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:10:00.806+0000 1694f700 -1 osd.3 369 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:10:00.935 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:10:00.928+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:25.641150+0000 front 2019-12-09T19:08:24.820519+0000 (oldest deadline 2019-12-09T19:09:10.483553+0000) 2019-12-09T19:10:00.940 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:10:00.933+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:31.151503+0000 front 2019-12-09T19:08:28.471080+0000 (oldest deadline 2019-12-09T19:09:09.347007+0000) 2019-12-09T19:10:00.972 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:10:00.967+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:29.092951+0000 front 2019-12-09T19:08:28.861461+0000 (oldest deadline 2019-12-09T19:09:09.920815+0000) 2019-12-09T19:10:01.216 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:10:01.210+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 45sec e369 v369) 2019-12-09T19:10:01.549 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:10:01.544+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:27.869010+0000 front 2019-12-09T19:08:26.850206+0000 (oldest deadline 2019-12-09T19:09:11.599638+0000) 2019-12-09T19:10:01.608 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:10:01.605+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:08:24.649762+0000 front 2019-12-09T19:08:29.619297+0000 (oldest deadline 2019-12-09T19:09:09.558825+0000) 2019-12-09T19:10:01.610 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:10:01.607+0000 1694f700 -1 osd.4 369 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:10:01.841 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:10:01.815+0000 1694f700 -1 osd.3 370 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e367) 2019-12-09T19:10:02.627 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:10:02.620+0000 1694f700 -1 osd.4 370 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:126684 5.3 5:c609908c:::notify.5:head [watch ping cookie 5402667728 gen 2] snapc 0=[] ondisk+write+known_if_redirected e369) 2019-12-09T19:10:35.051 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:27:31.670 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.057 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:27:31.676 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.072 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:27:31.691 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.072 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:27:31.671 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.077 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:27:31.676 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.077 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:27:31.696 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.385 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:27:32.004 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.394 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:27:31.993 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:35.399 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:27:32.017 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:56.152 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:11:56.145+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:11:56.183 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:11:56.176+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:11:56.636 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:11:56.630+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:11:57.165 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:11:57.159+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:11:57.223 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:11:57.218+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:11:57.601 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:11:57.596+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:11:57.606 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:11:57.601+0000 1694f700 -1 osd.7 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.279777+0000 front 2019-12-09T19:11:13.709644+0000 (oldest deadline 2019-12-09T19:11:56.722326+0000) 2019-12-09T19:11:58.205 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:11:58.199+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:11:58.233 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:11:58.227+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:11:58.639 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:11:58.633+0000 1694f700 -1 osd.7 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.279777+0000 front 2019-12-09T19:11:13.709644+0000 (oldest deadline 2019-12-09T19:11:56.722326+0000) 2019-12-09T19:11:58.640 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:11:58.633+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:11:59.209 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:11:59.203+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:11:59.212 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:11:59.206+0000 1694f700 -1 osd.2 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.277714+0000 front 2019-12-09T19:11:14.276059+0000 (oldest deadline 2019-12-09T19:11:58.432333+0000) 2019-12-09T19:11:59.257 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:11:59.251+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:11:59.329 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:11:59.322+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:11:59.625 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:11:59.614+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:00.247 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:00.242+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:00.266 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:12:00.260+0000 1694f700 -1 osd.2 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.277714+0000 front 2019-12-09T19:11:14.276059+0000 (oldest deadline 2019-12-09T19:11:58.432333+0000) 2019-12-09T19:12:00.289 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:00.284+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:00.316 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:00.309+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:00.654 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:00.647+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:00.967 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:00.961+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:01.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:01.227+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:01.274 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:01.268+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:01.353 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:01.346+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:01.684 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:01.676+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:01.931 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:01.925+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:02.242 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:02.236+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:02.288 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:02.281+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:02.392 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:02.384+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:02.719 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:02.713+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:02.900 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:02.895+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:03.229 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:03.223+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:03.252 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:03.247+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:03.363 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:03.353+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:03.682 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:03.676+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:03.862 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:03.856+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:04.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:04.241+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:04.278 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:04.272+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:04.315 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:04.308+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:04.658 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:04.652+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:04.908 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:04.902+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:05.234 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:05.229+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:05.274 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:05.269+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:05.287 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:05.282+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:05.647 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:05.641+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:05.904 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:05.898+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:06.216 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:06.210+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:06.269 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:06.264+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:06.300 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:06.289+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:06.614 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:06.608+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:06.917 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:06.911+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:07.202 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:07.193+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:07.243 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:07.238+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:07.287 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:07.281+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:07.617 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:07.611+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:07.892 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:07.886+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:08.198 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:08.193+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:08.238 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:08.232+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:08.325 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:08.319+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:08.623 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:08.616+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:08.903 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:08.897+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:09.231 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:09.225+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:09.275 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:09.268+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:09.355 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:09.350+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:09.609 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:09.604+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:09.909 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:09.904+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:10.234 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:10.228+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:10.288 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:10.283+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:10.340 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:10.334+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:10.584 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:10.577+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:10.890 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:10.884+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:11.229 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:11.223+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:11.266 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:11.260+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:11.308 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:11.302+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:11.627 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:11.620+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:11.918 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:11.912+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:12.203 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:12.199+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:12.298 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:12.291+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:12.315 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:12.308+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:12.583 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:12.576+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:12.945 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:12.939+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:13.221 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:13.215+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:13.317 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:13.311+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:13.372 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:13.360+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:13.551 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:13.541+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:13.901 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:13.896+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:14.183 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:14.179+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:14.341 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:14.332+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:14.397 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:14.390+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:14.512 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:14.507+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:14.889 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:14.883+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:15.196 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:15.191+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:15.369 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:15.356+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:15.395 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:15.389+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:15.555 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:15.549+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:15.865 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:15.858+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:16.160 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:16.155+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:16.391 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:16.385+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:16.401 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:16.397+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:16.554 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:16.549+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:16.910 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:16.906+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:17.156 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:17.151+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:17.407 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:17.403+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:17.442 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:17.437+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:17.565 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:17.561+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:17.927 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:17.922+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:18.194 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:18.189+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:18.424 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:18.419+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:18.451 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:18.442+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:18.542 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:18.537+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:18.976 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:18.971+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:19.167 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:19.161+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:19.482 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:19.476+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:19.501 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:19.495+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:19.553 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:19.548+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:19.962 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:19.957+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:20.207 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:20.201+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:20.500 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:20.494+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:20.520 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:20.514+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:20.557 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:20.543+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:20.925 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:20.920+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:21.224 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:21.217+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:21.487 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:21.481+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:21.496 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:21.490+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:21.514 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:21.507+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:21.910 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:21.906+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:22.272 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:22.264+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:22.474 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:22.470+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:22.520 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:22.515+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:22.551 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:22.545+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:22.940 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:22.934+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:23.241 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:23.234+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:23.433 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:23.428+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:23.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:23.505+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:23.586 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:23.578+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:23.983 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:23.979+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:24.293 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:24.285+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:24.431 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:24.426+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:24.488 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:24.484+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:24.565 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:24.557+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:24.971 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:24.967+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:25.304 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:25.298+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:25.411 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:25.406+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:25.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:25.511+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:25.535 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:25.528+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:25.969 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:25.963+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:26.335 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:26.328+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:26.397 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:26.391+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:26.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:26.489+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:26.570 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:26.564+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:26.809 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:12:26.804+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e376 v376) 2019-12-09T19:12:26.927 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:26.921+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:27.334 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:27.327+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:27.448 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:27.442+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:27.509 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:27.503+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:27.610 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:27.603+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:27.937 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:27.927+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:28.305 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:28.298+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:28.474 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:28.468+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:28.523 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:28.517+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:28.662 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:28.656+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:28.895 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:28.890+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:29.330 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:29.324+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:29.487 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:29.478+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:29.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:29.487+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:29.654 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:29.648+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:29.938 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:29.933+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:30.363 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:30.358+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:30.472 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:30.466+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:30.475 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:30.469+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:30.659 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:30.652+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:30.967 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:30.961+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:31.413 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:31.407+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:31.456 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:31.450+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:31.506 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:31.495+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:31.709 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:31.703+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:31.819 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:12:31.813+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e376 v376) 2019-12-09T19:12:32.005 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:31.999+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:32.404 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:32.397+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:32.469 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:32.463+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:32.517 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:32.511+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:32.700 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:32.694+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:33.041 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:33.033+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:33.416 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:33.410+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:33.476 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:33.470+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:33.566 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:33.560+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:33.688 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:33.680+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:34.078 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:34.072+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:34.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:34.394+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:34.512 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:34.506+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:34.538 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:34.531+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:34.665 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:34.658+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:35.103 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:35.097+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:35.358 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:35.351+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:35.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:35.488+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:35.583 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:35.572+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:35.705 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:35.699+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:36.069 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:12:36.063+0000 1694f700 -1 osd.5 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.668156+0000 front 2019-12-09T19:11:14.248517+0000 (oldest deadline 2019-12-09T19:12:00.111073+0000) 2019-12-09T19:12:36.347 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:12:36.340+0000 1694f700 -1 osd.1 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.429928+0000 front 2019-12-09T19:11:15.395273+0000 (oldest deadline 2019-12-09T19:11:55.317398+0000) 2019-12-09T19:12:36.496 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:12:36.490+0000 1694f700 -1 osd.4 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.673230+0000 front 2019-12-09T19:11:14.320784+0000 (oldest deadline 2019-12-09T19:11:58.346494+0000) 2019-12-09T19:12:36.552 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:12:36.547+0000 1694f700 -1 osd.6 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:11.080332+0000 front 2019-12-09T19:11:11.081933+0000 (oldest deadline 2019-12-09T19:11:56.022474+0000) 2019-12-09T19:12:36.706 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:12:36.699+0000 1694f700 -1 osd.0 376 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:11:14.133095+0000 front 2019-12-09T19:11:13.747194+0000 (oldest deadline 2019-12-09T19:11:55.749259+0000) 2019-12-09T19:12:36.881 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:12:36.874+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 42sec e376 v376) 2019-12-09T19:13:12.115 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:30:08.733 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.130 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:30:08.749 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.132 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:30:08.751 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.145 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:30:08.764 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.175 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:30:08.774 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.181 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:30:08.780 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.268 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:30:08.887 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.270 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:30:08.889 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:12.336 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:30:08.935 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:13:35.737 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:35.732+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:36.069 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:36.063+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:36.725 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:36.720+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:37.060 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:37.054+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:37.729 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:37.724+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:38.096 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:38.091+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:38.191 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:38.186+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:38.632 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:38.626+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:38.723 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:38.718+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:38.768 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:38.762+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:38.772 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:38.766+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:39.109 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:39.104+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:39.224 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:39.218+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:39.617 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:39.607+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:39.755 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:39.749+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:39.771 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:39.761+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:39.822 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:39.815+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:40.119 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:40.113+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:40.274 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:40.267+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:40.596 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:40.590+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:40.733 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:40.727+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:40.791 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:40.784+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:40.791 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:40.786+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:41.143 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:41.138+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:41.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:41.225+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:41.652 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:41.645+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:41.715 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:41.709+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:41.759 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:41.754+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:41.797 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:41.791+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:42.168 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:42.162+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:42.226 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:42.221+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:42.678 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:42.671+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:42.702 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:42.697+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:42.714 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:42.710+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:42.844 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:42.837+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:43.205 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:43.199+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:43.235 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:43.229+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:43.649 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:43.644+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:43.671 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:43.666+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:43.710 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:43.705+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:43.841 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:43.835+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:44.160 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:44.154+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:44.204 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:44.197+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:44.636 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:44.629+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:44.714 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:44.705+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:44.722 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:44.718+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:44.847 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:44.841+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:45.149 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:45.143+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:45.249 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:45.243+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:45.621 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:45.614+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:45.747 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:45.742+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:45.759 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:45.753+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:45.805 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:45.799+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:45.833 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:45.826+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:46.114 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:46.108+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:46.208 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:46.202+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:46.594 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:46.588+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:46.746 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:46.740+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:46.764 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:46.758+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:46.770 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:46.765+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:46.840 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:46.834+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:47.152 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:47.148+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:47.188 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:47.182+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:47.601 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:47.596+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:47.743 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:47.740+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:47.757 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:47.750+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:47.797 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:47.790+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:47.872 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:47.867+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:48.144 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:48.139+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:48.240 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:48.232+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:48.566 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:48.560+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:48.713 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:48.705+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:48.778 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:48.774+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:48.797 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:48.792+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:48.863 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:48.857+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:49.103 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:49.098+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:49.203 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:49.196+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:49.570 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:49.564+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:49.719 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:49.714+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:49.775 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:49.769+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:49.820 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:49.816+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:49.831 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:49.825+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:50.062 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:50.058+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:50.226 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:50.217+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:50.628 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:50.621+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:50.722 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:50.718+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:50.820 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:50.814+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:50.840 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:50.834+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:50.871 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:50.866+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:51.077 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:51.072+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:51.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:51.240+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:51.591 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:51.580+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:51.771 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:51.767+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:51.853 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:51.846+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:51.863 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:51.856+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:51.864 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:51.858+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:51.891 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:51.887+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:52.044 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:52.040+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:52.267 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:52.261+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:52.607 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:52.601+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:52.778 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:52.774+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:52.827 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:52.820+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:52.909 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:52.903+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:52.913 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:52.907+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:52.930 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:52.926+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:53.026 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:53.022+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:53.225 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:53.219+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:53.656 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:53.649+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:53.819 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:53.814+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:53.846 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:53.840+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:53.882 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:53.879+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:53.886 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:53.881+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:53.890 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:53.884+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:54.038 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:54.033+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:54.183 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:54.177+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:54.654 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:54.647+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:54.813 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:54.808+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:54.865 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:54.861+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:54.866 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:54.859+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:54.902 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:54.892+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:54.904 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:54.894+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:55.033 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:55.025+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:55.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:55.191+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:55.669 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:55.663+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:55.834 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:55.830+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:55.863 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:55.857+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:55.907 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:55.901+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:55.918 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:55.910+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:55.919 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:55.912+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:56.083 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:56.078+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:56.250 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:56.244+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:56.686 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:56.679+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:56.849 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:56.843+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:56.925 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:56.902+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:56.925 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:56.906+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:56.926 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:56.899+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:56.961 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:56.947+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:57.114 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:57.108+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:57.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:57.239+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:57.696 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:57.690+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:57.868 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:57.863+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:57.909 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:57.904+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:57.934 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:57.928+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:57.942 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:57.935+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:57.942 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:57.936+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:58.091 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:58.085+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:58.251 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:58.244+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:58.694 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:58.689+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:58.865 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:58.860+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:58.878 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:58.873+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:58.900 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:58.895+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:13:58.970 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:58.963+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:58.971 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:58.965+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:59.119 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:13:59.114+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:13:59.210 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:13:59.203+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:13:59.656 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:13:59.649+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:13:59.879 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:13:59.874+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:13:59.928 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:59.921+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:13:59.928 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:13:59.922+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:13:59.930 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:13:59.926+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:13:59.939 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:13:59.933+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:00.107 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:00.101+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:00.252 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:00.242+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:00.693 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:00.687+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:00.846 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:00.840+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:00.943 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:00.923+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:00.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:00.938+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:00.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:00.939+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:00.989 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:00.970+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:01.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:01.125+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:01.248 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:01.242+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:01.679 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:01.673+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:01.877 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:01.871+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:01.921 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:01.912+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:01.973 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:01.969+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:01.997 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:01.991+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:02.000 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:01.995+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:02.101 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:02.095+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:02.263 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:02.257+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:02.708 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:02.700+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:02.892 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:02.886+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:02.919 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:02.914+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:02.957 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:02.951+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:02.963 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:02.955+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:02.966 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:02.960+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:03.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:03.130+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:03.272 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:03.267+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:03.763 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:03.751+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:03.934 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:03.927+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:03.960 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:03.954+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:03.964 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:03.959+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:04.007 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:03.996+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:04.007 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:04.000+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:04.113 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:04.109+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:04.246 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:04.240+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:04.728 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:04.721+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:04.905 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:04.899+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:04.974 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:04.970+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:04.976 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:04.970+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:04.977 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:04.972+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:04.977 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:04.971+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:05.122 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:05.118+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:05.243 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:05.234+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:05.768 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:05.759+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:05.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:05.871+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:05.942 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:05.935+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:05.943 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:05.937+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:05.976 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:05.971+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:05.980 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:05.975+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:06.097 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:06.092+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:06.285 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:06.279+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:06.783 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:06.777+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:06.840 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T19:14:06.822+0000 20e25700 -1 RGWWatcher::handle_error cookie 363248400 err (107) Transport endpoint is not connected 2019-12-09T19:14:06.854 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:06.847+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:06.956 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:06.951+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:06.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:06.953+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:06.972 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:06.966+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:07.005 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:07.000+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:07.067 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:07.061+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:07.282 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:14:07.263+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 47sec e382 v382) 2019-12-09T19:14:07.328 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:07.322+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:07.802 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:07.796+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:07.877 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:07.872+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:07.967 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:07.961+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:07.968 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:07.962+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:07.987 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:07.982+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:08.053 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:08.047+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:08.089 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:08.084+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:08.374 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:08.369+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:08.773 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:08.767+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:08.834 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:08.828+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:08.942 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:08.937+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:08.942 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:08.938+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:08.981 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:08.976+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:08.983 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:08.978+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:09.064 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:09.058+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:09.081 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:09.075+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:09.428 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:09.421+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:09.823 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:09.816+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:09.829 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:09.824+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:09.901 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:09.895+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:09.902 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:09.897+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:10.010 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:10.004+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:10.013 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:10.008+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:10.102 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:10.096+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:10.132 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:10.127+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:10.457 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:10.450+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:10.822 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:10.815+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:10.835 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:10.827+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:10.856 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:10.850+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:10.856 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:10.851+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:10.972 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:10.965+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:10.973 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:10.966+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:11.109 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:11.104+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:11.134 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:11.128+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:11.433 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:11.427+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:11.780 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:11.774+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:11.812 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:11.804+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:11.906 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:11.900+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:11.907 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:11.902+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:11.951 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:11.944+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:11.953 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:11.946+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:12.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:12.088+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:12.146 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:12.140+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:12.288 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:14:12.276+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 47sec e382 v382) 2019-12-09T19:14:12.421 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:12.415+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:12.736 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:12.730+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:12.837 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:12.831+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:12.902 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:12.896+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:12.902 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:12.898+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:12.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:12.988+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:12.995 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:12.990+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:13.139 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:13.134+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:13.170 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:13.165+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:13.451 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:13.445+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:13.777 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:13.770+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:13.854 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:13.847+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:13.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:13.927+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:13.935 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:13.931+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:13.958 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:13.950+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:13.959 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:13.953+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:14.130 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:14.126+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:14.135 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:14.131+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:14.495 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:14.488+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:14.771 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:14.766+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:14.826 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:14.813+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:14.917 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:14.911+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:14.918 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:14.913+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:14.986 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:14.979+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:14.986 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:14.981+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:15.127 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:15.122+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:15.156 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:15.151+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:15.479 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:15.472+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:15.737 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:15.730+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:15.783 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:15.776+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:15.902 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:15.896+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:15.903 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:15.898+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:15.993 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:15.986+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:15.994 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:15.988+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:16.125 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:16.119+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:16.142 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:16.136+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:16.517 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:16.511+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:16.752 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:16.745+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:16.752 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:16.747+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:16.933 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:16.922+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:16.933 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:16.924+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:17.015 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:17.008+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:17.019 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:17.013+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:17.109 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:17.096+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:17.166 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:17.160+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:17.300 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:14:17.292+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 47sec e382 v382) 2019-12-09T19:14:17.515 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:17.508+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:17.726 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:17.718+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:17.739 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:17.733+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:17.904 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:17.898+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:17.907 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:17.903+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:18.058 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:18.051+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:18.059 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:18.053+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:18.108 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:18.102+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:18.166 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:18.160+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:18.528 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:18.521+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:18.717 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:18.710+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:18.720 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:18.714+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:18.873 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:18.869+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:18.877 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:18.872+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:19.024 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:19.016+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:19.024 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:19.019+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:19.159 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:19.154+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:19.177 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:19.169+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:19.537 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:19.532+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:19.692 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:19.686+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:19.747 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:19.741+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:19.907 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:19.901+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:19.907 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:19.902+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:20.065 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:20.048+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:20.065 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:20.050+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:20.189 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:20.184+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:20.231 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:20.225+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:20.546 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:20.540+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:20.717 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:20.711+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:20.770 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:20.763+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:20.869 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:20.864+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:20.870 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:20.865+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:21.090 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:21.083+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:21.091 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:21.084+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:21.214 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:21.208+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:21.283 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:21.278+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:21.588 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:21.579+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:21.685 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:21.677+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:21.733 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:21.727+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:21.899 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:21.894+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:21.902 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:21.898+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:22.053 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:22.045+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:22.053 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:22.046+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:22.170 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:22.165+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:22.319 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:14:22.313+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 47sec e382 v382) 2019-12-09T19:14:22.329 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:22.323+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:22.574 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:22.567+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:22.655 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:22.649+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:22.737 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:22.729+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:22.932 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:22.926+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:22.933 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:22.928+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:23.017 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:23.011+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:23.017 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:23.012+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:23.207 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:23.202+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:23.341 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:23.335+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:23.542 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:23.535+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:23.631 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:23.626+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:23.741 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:23.735+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:23.907 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:23.902+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:23.909 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:23.905+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:24.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:24.033+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:24.041 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:24.035+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:24.242 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:24.238+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:24.388 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:24.374+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:24.511 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:24.505+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:24.605 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:24.600+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:24.760 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:24.750+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:24.922 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:24.916+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:24.925 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:24.918+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:25.039 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:25.033+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:25.043 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:25.037+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:25.248 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:25.243+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:25.412 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:25.406+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:25.479 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:25.473+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:25.583 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:25.577+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:25.780 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:25.774+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:25.910 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:25.904+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:25.911 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:25.906+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:26.036 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:26.029+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:26.039 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:26.034+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:26.293 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:26.288+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:26.449 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:26.443+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:26.468 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:14:26.461+0000 1694f700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.960811+0000 front 2019-12-09T19:12:54.596377+0000 (oldest deadline 2019-12-09T19:13:37.886745+0000) 2019-12-09T19:14:26.571 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:14:26.565+0000 1694f700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:52.889271+0000 front 2019-12-09T19:12:53.230581+0000 (oldest deadline 2019-12-09T19:13:37.975676+0000) 2019-12-09T19:14:26.756 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:14:26.749+0000 1694f700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:49.594391+0000 front 2019-12-09T19:12:49.835869+0000 (oldest deadline 2019-12-09T19:13:35.014621+0000) 2019-12-09T19:14:26.884 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:26.877+0000 1694f700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:54.600999+0000 front 2019-12-09T19:12:54.410603+0000 (oldest deadline 2019-12-09T19:13:38.293522+0000) 2019-12-09T19:14:26.884 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:26.879+0000 1694f700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:27.049 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:27.043+0000 1694f700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:13:10.877773+0000 (oldest deadline 2019-12-09T19:13:50.877773+0000) 2019-12-09T19:14:27.052 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:27.047+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:27.304 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:14:27.298+0000 1694f700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.222559+0000 front 2019-12-09T19:12:54.448739+0000 (oldest deadline 2019-12-09T19:13:37.858539+0000) 2019-12-09T19:14:27.420 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:14:27.408+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 47sec e382 v382) 2019-12-09T19:14:27.476 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:14:27.470+0000 1694f700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:12:53.249456+0000 front 2019-12-09T19:12:52.042004+0000 (oldest deadline 2019-12-09T19:13:35.558638+0000) 2019-12-09T19:14:28.070 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:28.028+0000 1694f700 -1 osd.4 383 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:28.261 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:14:28.088+0000 1694f700 -1 osd.3 383 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e380) 2019-12-09T19:14:29.181 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:14:29.174+0000 1694f700 -1 osd.4 384 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127101 5.3 5:c609908c:::notify.5:head [watch ping cookie 363248400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e382) 2019-12-09T19:14:36.922 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:31:33.541 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:36.928 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:31:33.547 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:36.939 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:31:33.557 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:36.944 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:31:33.561 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:36.950 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:31:33.549 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:36.960 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:31:33.558 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:37.046 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:31:33.665 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:37.065 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:31:33.684 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:37.074 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:31:33.673 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:16:17.512 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:17.506+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:17.860 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:17.855+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:18.471 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:18.462+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:18.534 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:18.527+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:18.904 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:18.899+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:19.077 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:19.071+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:19.126 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:19.119+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:19.193 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:19.181+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:19.455 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:19.449+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:19.586 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:19.579+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:19.870 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:19.865+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:20.093 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:20.087+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:20.100 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:20.095+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:20.185 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:20.180+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:20.435 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:20.428+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:20.558 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:20.552+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:20.903 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:20.896+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:21.069 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:21.061+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:21.107 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:21.099+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:21.198 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:21.191+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:21.428 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:21.421+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:21.605 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:21.600+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:21.849 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:21.843+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:21.948 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:21.941+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:22.066 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:22.059+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:22.144 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:22.138+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:22.228 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:22.223+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:22.408 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:22.402+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:22.617 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:22.611+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:22.857 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:22.850+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:22.996 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:22.989+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:23.099 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:23.093+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:23.100 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:23.095+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:23.230 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:23.223+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:23.426 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:23.420+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:23.596 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:23.590+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:23.889 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:23.882+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:23.999 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:23.993+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:24.141 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:24.135+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:24.143 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:24.137+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:24.216 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:24.211+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:24.395 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:24.388+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:24.628 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:24.622+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:24.923 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:24.911+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:24.958 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:24.952+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:25.128 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:25.122+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:25.182 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:25.176+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:25.216 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:25.210+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:25.420 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:25.414+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:25.673 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:25.667+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:25.973 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:25.963+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:26.008 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:26.001+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:26.083 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:26.078+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:26.219 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:26.213+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:26.224 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:26.218+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:26.466 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:26.459+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:26.687 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:26.682+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:26.939 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:26.933+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:26.980 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:26.973+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:27.086 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:27.080+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:27.245 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:27.238+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:27.255 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:27.248+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:27.505 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:27.499+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:27.726 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:27.720+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:27.941 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:27.935+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:27.970 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:27.965+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:28.056 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:28.050+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:28.204 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:28.197+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:28.222 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:28.216+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:28.469 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:28.461+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:28.705 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:28.699+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:28.933 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:28.926+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:29.001 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:28.995+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:29.020 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:29.007+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:29.214 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:29.208+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:29.227 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:29.221+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:29.513 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:29.506+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:29.696 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:29.690+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:29.904 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:29.898+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:30.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:30.027+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:30.059 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:30.053+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:30.202 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:30.196+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:30.239 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:30.230+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:30.550 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:30.544+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:30.659 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:30.652+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:30.899 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:30.892+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:31.017 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:31.011+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:31.038 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:31.033+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:31.223 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:31.217+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:31.250 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:31.244+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:31.577 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:31.570+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:31.692 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:31.686+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:31.853 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:31.848+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:32.057 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:32.050+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:32.060 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:32.054+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:32.208 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:32.202+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:32.258 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:32.251+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:32.549 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:32.543+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:32.661 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:32.650+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:32.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:32.883+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:33.031 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:33.024+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:33.110 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:33.104+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:33.218 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:33.213+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:33.273 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:33.267+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:33.582 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:33.575+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:33.634 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:33.627+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:33.865 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:33.859+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:34.012 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:34.006+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:34.156 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:34.150+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:34.217 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:34.211+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:34.229 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:34.224+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:34.599 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:34.589+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:34.609 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:34.599+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:34.884 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:34.878+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:34.991 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:34.984+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:35.187 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:35.181+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:35.208 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:35.202+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:35.252 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:35.242+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:35.553 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:35.546+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:35.648 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:35.643+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:35.851 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:35.844+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:35.961 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:35.955+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:36.172 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:36.167+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:36.234 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:36.228+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:36.244 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:36.238+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:36.574 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:36.568+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:36.657 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:36.651+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:36.874 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:36.866+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:36.922 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:36.916+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:37.165 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:37.159+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:37.240 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:37.234+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:37.288 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:37.278+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:37.583 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:37.577+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:37.701 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:37.695+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:37.894 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:37.885+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:37.968 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:37.959+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:38.194 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:38.188+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:38.282 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:38.277+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:38.324 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:38.318+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:38.614 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:38.607+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:38.702 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:38.695+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:38.879 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:38.872+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:38.936 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:38.930+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:39.162 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:39.155+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:39.327 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:39.321+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:39.377 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:39.371+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:39.613 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:39.606+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:39.708 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:39.702+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:39.866 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:39.860+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:39.965 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:39.957+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:40.211 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:40.205+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:40.347 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:40.342+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:40.395 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:40.390+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:40.611 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:40.604+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:40.739 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:40.733+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:40.846 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:40.839+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:40.995 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:40.989+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:41.188 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:41.183+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:41.347 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:41.342+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:41.366 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:41.359+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:41.621 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:41.614+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:41.752 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:41.746+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:41.887 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:41.879+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:42.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:42.024+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:42.150 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:42.142+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:42.380 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:42.373+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:42.418 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:42.412+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:42.603 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:42.595+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:42.775 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:42.770+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:42.891 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:42.885+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:43.021 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:43.015+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:43.108 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:43.101+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:43.399 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:43.394+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:43.399 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:43.394+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:43.645 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:43.640+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:43.792 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:43.785+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:43.919 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:43.912+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:43.983 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:43.977+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:44.156 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:44.151+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:44.395 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:44.389+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:44.421 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:44.415+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:44.652 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:44.646+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:44.762 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:44.756+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:44.895 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:44.889+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:45.025 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:45.019+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:45.147 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:45.141+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:45.379 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:45.373+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:45.474 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:45.469+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:45.652 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:45.646+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:45.735 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:45.730+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:45.929 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:45.913+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:46.041 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:46.033+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:46.114 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:46.108+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:46.384 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:46.378+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:46.442 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:46.436+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:46.679 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:46.671+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:46.770 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:46.764+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:46.957 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:46.951+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:47.038 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:47.031+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:47.085 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:47.080+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:47.381 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:47.376+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:47.401 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:47.394+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:47.706 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:47.698+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:47.744 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:47.738+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:47.957 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:47.949+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:48.061 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:48.055+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:48.079 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:48.073+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:48.343 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:48.338+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:48.415 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:48.408+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:48.690 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:48.682+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:48.709 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:48.704+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:48.928 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:48.922+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:49.054 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:49.047+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:49.079 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:49.075+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:49.301 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:49.297+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:49.462 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:49.457+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:49.731 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:49.724+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:49.731 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:49.726+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:49.950 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:49.943+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:50.028 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:50.018+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:50.140 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:50.124+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:50.341 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:50.336+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:50.422 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:50.417+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:50.728 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:50.720+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:50.731 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:50.726+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:50.953 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:50.946+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:51.057 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:51.050+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:51.138 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:51.133+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:51.343 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:51.338+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:51.383 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:51.378+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:51.730 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:51.724+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:51.733 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:51.728+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:51.936 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:51.928+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:52.062 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:52.055+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:52.189 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:52.185+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:52.370 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:52.365+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:52.383 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:52.378+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:52.757 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:52.753+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:52.763 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:52.756+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:52.891 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:52.885+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:53.027 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:16:53.022+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e388 v388) 2019-12-09T19:16:53.103 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:53.096+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:53.159 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:53.155+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:53.384 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:53.379+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:53.420 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:53.417+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:53.729 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:53.723+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:53.734 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:53.728+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:53.889 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:53.876+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:54.145 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:54.138+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:54.192 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:54.187+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:54.376 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:54.371+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:54.467 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:54.462+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:54.719 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:54.707+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:54.728 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:54.724+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:54.881 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:54.875+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:55.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:55.095+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:55.177 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:55.172+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:55.363 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:55.358+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:55.479 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:55.474+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:55.689 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:55.684+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:55.728 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:55.721+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:55.846 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:55.839+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:56.071 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:56.064+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:56.142 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:56.136+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:56.394 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:56.389+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:56.522 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:56.517+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:56.675 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:56.670+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:56.722 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:56.701+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:56.859 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:56.851+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:57.121 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:57.113+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:57.161 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:57.156+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:57.384 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:57.380+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:57.548 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:57.543+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:57.699 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:57.693+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:57.723 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:57.717+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:57.904 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:57.897+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:58.041 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:16:58.037+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e388 v388) 2019-12-09T19:16:58.144 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:58.138+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:58.170 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:58.163+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:58.433 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:58.427+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:58.581 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:58.575+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:58.677 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:58.671+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:58.684 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:58.678+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:58.933 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:58.927+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:16:59.146 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:16:59.141+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:16:59.185 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:16:59.180+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:16:59.476 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:16:59.470+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:16:59.635 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:16:59.629+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:16:59.649 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:16:59.643+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:16:59.657 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:16:59.649+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:16:59.974 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:16:59.968+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:17:00.104 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:00.098+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:17:00.188 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:17:00.181+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:17:00.477 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:17:00.471+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:17:00.637 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:17:00.631+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:17:00.677 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:17:00.671+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:17:00.690 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:17:00.677+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:17:00.983 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:17:00.975+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:17:01.120 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:01.115+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:17:01.212 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:17:01.205+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:17:01.480 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:17:01.474+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:17:01.608 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:17:01.602+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:17:01.664 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:17:01.657+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:17:01.676 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:17:01.668+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:17:01.955 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:17:01.948+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:17:02.139 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:02.131+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:17:02.181 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:17:02.174+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:17:02.509 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:17:02.503+0000 1694f700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.434027+0000 front 2019-12-09T19:15:37.436004+0000 (oldest deadline 2019-12-09T19:16:18.976552+0000) 2019-12-09T19:17:02.615 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:17:02.608+0000 1694f700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.434713+0000 front 2019-12-09T19:15:35.361132+0000 (oldest deadline 2019-12-09T19:16:18.697627+0000) 2019-12-09T19:17:02.685 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:17:02.680+0000 1694f700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:36.952702+0000 front 2019-12-09T19:15:37.449304+0000 (oldest deadline 2019-12-09T19:16:17.380432+0000) 2019-12-09T19:17:02.722 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:17:02.716+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:17:02.922 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:17:02.916+0000 1694f700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.421925+0000 front 2019-12-09T19:15:37.422813+0000 (oldest deadline 2019-12-09T19:16:17.518700+0000) 2019-12-09T19:17:03.117 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:17:03.108+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e388 v388) 2019-12-09T19:17:03.126 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:03.116+0000 1694f700 -1 osd.4 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.337795+0000 front 2019-12-09T19:15:34.208485+0000 (oldest deadline 2019-12-09T19:16:18.221624+0000) 2019-12-09T19:17:03.138 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:17:03.132+0000 1694f700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:37.424101+0000 front 2019-12-09T19:15:37.422943+0000 (oldest deadline 2019-12-09T19:16:21.245319+0000) 2019-12-09T19:17:03.681 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:17:03.676+0000 1694f700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:15:35.439634+0000 front 2019-12-09T19:15:36.349036+0000 (oldest deadline 2019-12-09T19:16:18.321973+0000) 2019-12-09T19:17:38.466 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.291+0000 34a75700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:38.551 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.330+0000 3827c700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:38.634 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.331+0000 34a75700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:38.723 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.358+0000 35276700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:38.730 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.389+0000 39a7f700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:38.739 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.427+0000 3827c700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:39.016 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:38.929+0000 36a79700 -1 osd.4 393 build_incremental_map_msg missing incremental map 393 2019-12-09T19:17:41.644 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:41.616+0000 39a7f700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:41.752 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:41.742+0000 35a77700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:41.812 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:41.775+0000 37a7b700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:41.889 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:41.862+0000 3727a700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:41.914 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:41.865+0000 37a7b700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:42.479 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:42.093+0000 3727a700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:42.479 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:42.168+0000 37a7b700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:45.039 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:17:45.002+0000 36a79700 -1 osd.3 395 build_incremental_map_msg missing incremental map 395 2019-12-09T19:17:53.650 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:34:50.268 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.689 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:34:50.306 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.754 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:34:50.373 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.756 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:34:50.375 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.760 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:34:50.359 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.762 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:34:50.361 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.958 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:34:50.553 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:53.978 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:34:50.597 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:54.023 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:34:50.642 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:17:54.048 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T19:17:54.013+0000 20e25700 -1 RGWWatcher::handle_error cookie 5440092720 err (107) Transport endpoint is not connected 2019-12-09T19:17:55.017 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:55.012+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:17:55.989 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:55.985+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:17:57.021 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:57.006+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:17:58.007 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:58.002+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:17:59.027 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:17:59.023+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:17:59.295 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:17:59.284+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:00.020 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:00.014+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:00.271 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:00.265+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:00.730 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:00.725+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:00.833 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:00.824+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:00.994 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:00.989+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:01.288 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:01.280+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:01.730 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:01.722+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:01.808 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:01.802+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:01.978 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:01.974+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:02.086 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:02.080+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:02.241 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:02.236+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:02.649 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:02.643+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:02.775 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:02.769+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:02.859 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:02.853+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:03.033 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:03.019+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:03.129 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:03.123+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:03.230 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:03.226+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:03.663 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:03.658+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:03.811 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:03.805+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:03.904 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:03.898+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:04.034 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:04.029+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:04.103 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:04.097+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:04.235 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:04.230+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:04.667 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:04.662+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:04.848 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:04.842+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:04.879 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:04.873+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:05.010 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:05.005+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:05.084 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:05.076+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:05.260 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:05.254+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:05.638 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:05.633+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:05.890 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:05.883+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:05.897 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:05.891+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:05.989 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:05.982+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:05.990 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:05.985+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:06.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:06.094+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:06.276 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:06.271+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:06.632 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:06.626+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:06.908 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:06.899+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:06.945 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:06.928+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:06.962 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:06.945+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:06.962 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:06.947+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:07.119 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:07.110+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:07.236 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:07.230+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:07.611 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:07.605+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:07.926 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:07.920+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:07.961 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:07.955+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:07.964 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:07.958+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:07.965 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:07.960+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:08.083 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:08.077+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:08.221 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:08.216+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:08.644 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:08.637+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:08.900 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:08.894+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:08.927 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:08.921+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:08.927 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:08.923+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:08.988 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:08.981+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:09.095 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:09.089+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:09.243 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:09.237+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:09.673 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:09.668+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:09.868 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:09.858+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:09.922 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:09.916+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:09.923 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:09.918+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:10.028 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:10.022+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:10.077 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:10.071+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:10.237 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:10.232+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:10.248 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:10.243+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:10.716 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:10.710+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:10.867 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:10.862+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:10.878 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:10.871+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:10.878 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:10.873+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:11.055 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:11.049+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:11.104 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:11.097+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:11.266 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:11.260+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:11.267 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:11.261+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:11.694 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:11.690+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:11.829 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:11.823+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:11.859 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:11.854+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:11.860 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:11.855+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:12.039 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:12.032+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:12.141 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:12.132+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:12.231 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:12.225+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:12.278 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:12.272+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:12.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:12.727+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:12.809 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:12.799+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:12.849 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:12.843+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:12.850 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:12.845+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:13.005 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:13.000+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:13.103 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:13.096+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:13.258 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:13.252+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:13.294 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:13.287+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:13.757 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:13.752+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:13.780 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:13.774+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:13.901 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:13.894+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:13.901 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:13.896+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:14.009 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:14.003+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:14.118 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:14.112+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:14.230 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:14.224+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:14.312 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:14.307+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:14.727 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:14.723+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:14.831 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:14.825+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:14.859 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:14.853+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:14.860 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:14.855+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:14.998 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:14.992+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:15.158 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:15.152+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:15.217 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:15.211+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:15.281 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:15.275+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:15.730 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:15.724+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:15.814 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:15.808+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:15.863 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:15.857+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:15.864 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:15.859+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:16.017 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:16.011+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:16.138 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:16.132+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:16.200 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:16.193+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:16.201 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:16.195+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:16.245 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:16.240+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:16.699 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:16.694+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:16.793 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:16.786+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:16.895 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:16.889+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:16.896 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:16.891+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:17.063 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:17.057+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:17.101 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:17.087+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:17.198 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:17.193+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:17.206 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:17.201+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:17.209 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:17.204+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:17.682 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:17.677+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:17.842 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:17.836+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:17.881 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:17.875+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:17.882 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:17.877+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:18.062 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:18.055+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:18.099 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:18.093+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:18.223 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:18.218+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:18.251 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:18.244+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:18.252 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:18.246+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:18.659 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:18.654+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:18.810 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:18.804+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:18.912 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:18.907+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:18.915 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:18.911+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:19.075 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:19.068+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:19.109 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:19.102+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:19.240 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:19.230+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:19.241 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:19.233+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:19.266 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:19.260+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:19.632 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:19.628+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:19.820 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:19.813+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:19.955 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:19.950+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:19.961 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:19.957+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:20.046 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:20.040+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:20.150 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:20.142+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:20.208 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:20.201+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:20.209 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:20.203+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:20.262 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:20.258+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:20.634 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:20.629+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:20.812 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:20.807+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:20.999 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:20.995+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:21.001 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:20.997+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:21.018 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:21.013+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:21.178 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:21.173+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:21.200 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:21.194+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:21.202 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:21.197+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:21.241 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:21.236+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:21.607 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:21.602+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:21.804 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:21.798+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:21.969 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:21.964+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:21.975 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:21.969+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:22.026 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:22.015+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:22.175 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:22.169+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:22.194 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:22.188+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:22.197 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:22.191+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:22.282 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:22.278+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:22.626 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:22.622+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:22.779 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:22.772+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:22.935 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:22.931+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:22.936 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:22.932+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:22.998 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:22.993+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:23.171 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:23.163+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:23.187 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:23.181+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:23.190 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:23.185+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:23.254 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:23.250+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:23.651 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:23.647+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:23.776 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:23.770+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:23.945 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:23.941+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:23.946 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:23.942+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:23.991 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:23.985+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:24.177 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:24.170+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:24.215 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:24.211+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:24.240 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:24.235+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:24.243 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:24.238+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:24.693 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:24.684+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:24.738 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:24.728+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:24.942 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:24.936+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:24.943 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:24.938+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:24.973 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:24.968+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:25.154 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:25.148+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:25.251 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:25.246+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:25.253 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:25.247+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:25.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:25.251+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:25.713 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:25.709+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:25.786 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:25.780+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:25.909 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:25.900+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:25.909 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:25.903+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:25.944 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:25.937+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:26.197 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:26.191+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:26.289 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:26.284+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:26.291 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:26.287+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:26.297 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:26.293+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:26.712 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:26.707+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:26.784 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:26.777+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:26.906 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:26.901+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:26.909 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:26.902+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:26.987 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:26.982+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:27.247 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:27.237+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:27.274 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:27.268+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:27.311 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:27.304+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:27.319 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:27.305+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:27.690 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:27.687+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:27.822 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:27.816+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:27.911 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:27.906+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:27.912 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:27.908+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:27.970 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:27.964+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:28.229 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:28.224+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:28.293 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:28.285+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:28.323 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:28.316+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:28.323 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:28.317+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:28.725 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:28.720+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:28.837 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:28.832+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:28.942 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:28.939+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:28.943 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:28.939+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:29.011 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:29.005+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:29.262 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:29.257+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:29.318 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:29.312+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:29.321 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:29.316+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:29.330 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:29.324+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:29.723 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:29.717+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:29.852 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:29.847+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:29.994 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:29.989+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:29.999 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:29.994+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:30.033 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:30.026+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:30.307 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:30.302+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:30.318 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:30.312+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:30.319 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:30.313+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:30.349 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:30.340+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:30.760 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:30.755+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:30.850 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:30.843+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:30.984 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:30.979+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:30.985 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:30.981+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:31.005 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:30.999+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:31.287 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:31.280+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:31.288 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:31.282+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:31.328 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:31.323+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:31.385 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:31.377+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:31.730 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:31.725+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:31.810 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:31.804+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:31.984 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:31.974+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:31.985 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:31.976+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:31.985 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:31.977+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:32.283 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:32.276+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:32.283 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:32.277+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:32.332 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:32.327+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:32.427 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:32.421+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:32.759 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:32.754+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:32.860 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:32.848+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:32.950 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:32.944+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:32.954 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:32.949+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:32.981 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:32.974+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:33.279 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:33.273+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:33.282 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:33.277+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:33.338 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:33.333+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:33.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:33.408+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:33.756 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:18:33.746+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e395 v395) 2019-12-09T19:18:33.756 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:33.751+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:33.821 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:33.814+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:33.968 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:33.962+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:33.969 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:33.963+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:33.988 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:33.983+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:34.306 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:34.298+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:34.307 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:34.300+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:34.317 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:34.305+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:34.396 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:34.390+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:34.748 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:34.744+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:34.833 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:34.828+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:34.987 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:34.982+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:34.990 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:34.985+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:35.030 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:35.025+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:35.185 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:35:31.804 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.194 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:35:31.813 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.213 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:35:31.832 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.214 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:35:31.813 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.219 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:35:31.838 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.226 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:35:31.825 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.296 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:35.291+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:35.297 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:35:31.915 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.318 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:35:31.937 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.353 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:35.348+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:35.353 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:35.349+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:35.368 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:35:31.966 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:35.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:35.394+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:35.755 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:35.750+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:35.881 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:35.877+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:35.945 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:35.940+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:35.949 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:35.945+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:36.000 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:35.996+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:36.316 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:36.311+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:36.319 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:36.315+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:36.332 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:36.328+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:36.397 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:36.392+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:36.751 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:36.744+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:36.932 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:36.928+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:36.946 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:36.940+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:36.948 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:36.942+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:37.010 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:37.006+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:37.297 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:37.292+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:37.298 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:37.293+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:37.307 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:37.301+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:37.447 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:37.442+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:37.726 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:37.721+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:37.968 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:37.962+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:37.996 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:37.990+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:37.996 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:37.992+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:38.010 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:38.003+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:38.346 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:38.341+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:38.346 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:38.341+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:38.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:38.345+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:38.480 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:38.475+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:38.777 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:38.772+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:38.781 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:18:38.771+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e395 v395) 2019-12-09T19:18:38.967 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:38.962+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:38.968 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:38.963+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:39.005 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:39.000+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:39.044 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:39.040+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:39.386 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:39.378+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:39.400 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:39.393+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:39.400 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:39.395+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:39.468 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:39.463+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:39.780 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:39.775+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:39.981 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:39.968+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:39.981 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:39.972+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:40.039 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:40.034+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:40.056 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:40.051+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:40.425 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:40.419+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:40.437 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:40.430+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:40.437 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:40.432+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:40.443 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:40.437+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:40.735 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:40.729+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:40.949 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:40.942+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:40.961 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:40.944+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:41.029 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:41.022+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:41.094 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:41.088+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:41.398 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:41.393+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:41.399 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:41.394+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:41.421 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:41.415+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:41.486 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:41.482+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:41.706 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:41.701+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:41.982 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:41.977+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:41.990 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:41.981+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:42.024 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:42.018+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:42.127 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:42.123+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:42.396 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:42.390+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:42.396 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:42.391+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:42.449 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:42.442+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:42.515 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:42.509+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:42.741 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:42.736+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:42.990 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:42.985+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:42.994 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:42.989+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:43.000 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:42.995+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:43.115 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:43.110+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:43.394 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:43.388+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:43.394 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:43.390+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:43.455 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:43.450+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:43.511 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:43.505+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:43.711 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:43.706+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:43.785 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:18:43.781+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e395 v395) 2019-12-09T19:18:43.977 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:43.973+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:43.988 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:43.982+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:43.988 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:43.984+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:44.080 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:44.075+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:44.425 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:44.420+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:44.426 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:44.421+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:44.430 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:44.423+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:44.527 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:44.521+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:44.697 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:44.691+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:44.957 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:44.953+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:44.982 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:44.977+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:44.987 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:44.982+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:45.074 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:45.069+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:45.420 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:45.414+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:45.421 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:45.416+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:45.473 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:45.467+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:45.497 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:45.493+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:45.721 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:45.715+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:45.971 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:45.965+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:46.037 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:46.032+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:46.044 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:46.036+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:46.058 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:46.052+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:46.421 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:46.416+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:46.422 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:46.417+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:46.516 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:46.511+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:46.532 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:46.527+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:46.755 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:46.745+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:46.950 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:46.946+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:47.016 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:47.010+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:47.017 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:47.012+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:47.116 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:47.105+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:47.403 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:47.397+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:47.404 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:47.398+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:47.506 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:47.501+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:47.573 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:47.561+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:47.772 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:47.767+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:47.983 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:47.971+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:47.983 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:47.974+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:47.992 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:47.987+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:48.149 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:18:48.144+0000 1694f700 -1 osd.2 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.411310+0000 front 2019-12-09T19:17:20.586481+0000 (oldest deadline 2019-12-09T19:18:00.369826+0000) 2019-12-09T19:18:48.426 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:48.421+0000 1694f700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:17:35.559428+0000 (oldest deadline 2019-12-09T19:18:15.559428+0000) 2019-12-09T19:18:48.429 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:48.425+0000 1694f700 -1 osd.3 395 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:48.473 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:48.467+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:48.568 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:18:48.562+0000 1694f700 -1 osd.0 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:16.222910+0000 front 2019-12-09T19:17:20.669832+0000 (oldest deadline 2019-12-09T19:18:00.088038+0000) 2019-12-09T19:18:48.731 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:18:48.726+0000 1694f700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.703492+0000 front 2019-12-09T19:17:20.906170+0000 (oldest deadline 2019-12-09T19:18:02.100382+0000) 2019-12-09T19:18:48.921 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:18:48.904+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 46sec e395 v395) 2019-12-09T19:18:48.978 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:48.969+0000 1694f700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.588790+0000 front 2019-12-09T19:17:20.884358+0000 (oldest deadline 2019-12-09T19:18:05.479550+0000) 2019-12-09T19:18:48.978 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:48.972+0000 1694f700 -1 osd.4 395 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:49.005 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:18:49.001+0000 1694f700 -1 osd.1 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:19.015351+0000 front 2019-12-09T19:17:19.844298+0000 (oldest deadline 2019-12-09T19:18:01.749767+0000) 2019-12-09T19:18:49.463 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:18:49.458+0000 1694f700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:17:20.913609+0000 front 2019-12-09T19:17:14.679090+0000 (oldest deadline 2019-12-09T19:17:58.630435+0000) 2019-12-09T19:18:49.758 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:49.503+0000 1694f700 -1 osd.3 396 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e393) 2019-12-09T19:18:49.977 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:18:49.966+0000 1694f700 -1 osd.4 396 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127460 5.3 5:c609908c:::notify.5:head [watch ping cookie 5440092720 gen 1] snapc 0=[] ondisk+write+known_if_redirected e391) 2019-12-09T19:18:50.012 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:18:49.956+0000 3727a700 -1 osd.3 396 build_incremental_map_msg missing incremental map 395 2019-12-09T19:19:15.479 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:19:15.272+0000 1694f700 -1 osd.6 392 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127432 6.1 6.c9be1e91 (undecoded) ondisk+read+known_if_redirected e391) 2019-12-09T19:19:25.057 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:19:24.974+0000 39a7f700 -1 osd.6 401 build_incremental_map_msg missing incremental map 401 2019-12-09T19:19:25.163 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:19:25.045+0000 34a75700 -1 osd.6 401 build_incremental_map_msg missing incremental map 401 2019-12-09T19:19:25.252 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:19:25.148+0000 38a7d700 -1 osd.6 401 build_incremental_map_msg missing incremental map 401 2019-12-09T19:19:29.745 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:19:29.737+0000 34274700 -1 osd.6 402 build_incremental_map_msg missing incremental map 401 2019-12-09T19:20:28.011 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:28.006+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:29.020 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:29.015+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:29.979 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:29.974+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:30.995 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:30.990+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:32.037 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:32.032+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:33.064 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:33.059+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:34.111 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:34.106+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:35.152 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:35.146+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:36.155 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:36.150+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:37.147 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:37.142+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:38.160 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:38.155+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:39.173 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:39.168+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:40.144 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:40.137+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:41.119 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:41.114+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:41.676 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:41.670+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:41.864 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:41.855+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:42.127 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:42.122+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:42.688 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:42.682+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:42.697 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:42.693+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:42.843 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:42.837+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:43.132 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:43.128+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:43.333 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:43.327+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:43.445 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:43.439+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:43.691 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:43.687+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:43.699 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:43.695+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:43.875 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:43.870+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:44.165 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:44.159+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:44.310 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:44.304+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:44.374 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:44.367+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:44.469 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:44.465+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:44.693 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:44.689+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:44.694 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:44.689+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:44.879 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:44.874+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:45.202 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:45.196+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:45.276 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:45.270+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:45.384 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:45.377+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:45.484 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:45.479+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:45.705 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:45.699+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:45.732 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:45.726+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:45.870 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:45.863+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:46.177 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:46.170+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:46.178 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:46.172+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:46.254 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:46.247+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:46.345 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:46.338+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:46.480 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:46.473+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:46.734 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:46.723+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:46.737 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:46.731+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:46.869 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:46.863+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:47.201 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:47.194+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:47.202 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:47.196+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:47.261 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:47.254+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:47.308 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:47.302+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:47.441 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:47.435+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:47.721 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:47.712+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:47.735 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:47.729+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:47.892 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:47.885+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:48.196 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:48.191+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:48.198 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:48.193+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:48.311 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:48.305+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:48.329 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:48.323+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:48.478 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:48.471+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:48.715 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:48.704+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:48.740 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:48.734+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:48.928 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:48.922+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:49.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:49.226+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:49.233 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:49.228+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:49.274 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:49.268+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:49.341 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:49.335+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:49.509 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:49.503+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:49.676 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:49.670+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:49.792 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:49.786+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:49.918 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:49.913+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:50.231 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:50.225+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:50.232 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:50.227+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:50.233 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:50.227+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:50.310 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:50.304+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:50.472 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:50.466+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:50.634 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:50.629+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:50.747 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:50.742+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:50.954 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:50.948+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:51.281 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:51.274+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:51.281 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:51.275+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:51.283 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:51.277+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:51.307 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:51.301+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:51.484 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:51.479+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:51.599 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:51.592+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:51.800 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:51.794+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:51.993 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:51.986+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:52.286 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:52.280+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:52.317 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:52.310+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:52.333 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:52.327+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:52.335 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:52.329+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:52.465 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:52.459+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:52.648 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:52.641+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:52.835 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:52.830+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:53.032 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:53.026+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:53.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:53.300+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:53.307 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:53.302+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:53.338 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:53.333+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:53.348 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:53.342+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:53.511 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:53.505+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:53.656 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:53.651+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:53.836 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:53.830+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:54.078 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:54.072+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:54.295 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:54.289+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:54.295 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:54.291+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:54.322 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:54.315+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:54.357 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:54.351+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:54.489 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:54.482+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:54.691 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:54.686+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:54.888 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:54.879+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:55.056 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:55.051+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:55.304 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:55.298+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:55.306 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:55.301+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:55.319 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:55.312+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:55.366 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:55.360+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:55.448 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:55.442+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:55.707 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:55.701+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:55.933 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:55.927+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:56.016 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:56.010+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:56.287 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:56.280+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:56.308 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:56.301+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:56.308 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:56.303+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:56.401 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:56.395+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:56.474 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:56.469+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:56.763 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:56.757+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:56.933 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:56.927+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:57.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:57.045+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:57.323 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:57.317+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:57.324 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:57.319+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:57.334 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:57.328+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:57.383 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:57.376+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:57.462 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:57.456+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:57.746 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:57.740+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:57.921 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:57.915+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:58.023 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:58.017+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:58.360 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:58.353+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:58.367 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:58.361+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:58.368 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:58.360+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:58.369 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:58.364+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:58.495 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:58.489+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:58.779 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:58.773+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:58.925 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:58.919+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:20:59.068 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:20:59.062+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:20:59.344 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:20:59.337+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:20:59.366 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:59.358+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:20:59.366 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:20:59.359+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:20:59.368 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:20:59.363+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:20:59.517 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:20:59.511+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:20:59.833 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:20:59.828+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:20:59.909 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:20:59.895+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:00.087 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:00.082+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:00.374 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:00.368+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:00.412 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:00.405+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:00.414 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:00.406+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:00.415 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:00.409+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:00.480 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:00.474+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:00.886 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:00.880+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:00.937 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:00.931+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:01.051 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:01.047+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:01.347 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:01.342+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:01.370 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:01.363+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:01.379 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:01.370+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:01.394 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:01.389+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:01.522 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:01.516+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:01.931 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:01.925+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:01.948 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:01.940+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:02.036 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:02.027+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:02.357 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:02.352+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:02.405 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:02.399+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:02.405 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:02.400+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:02.407 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:02.401+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:02.575 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:02.569+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:02.945 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:02.940+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:02.970 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:02.964+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:03.028 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:03.019+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:03.388 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:03.381+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:03.403 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:03.396+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:03.435 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:03.423+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:03.436 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:03.428+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:03.569 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:03.564+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:03.945 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:03.935+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:04.008 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:04.001+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:04.071 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:04.065+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:04.378 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:04.370+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:04.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:04.390+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:04.400 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:04.393+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:04.456 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:04.449+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:04.601 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:04.593+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:04.957 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:04.951+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:05.031 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:05.025+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:05.075 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:05.069+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:05.360 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:05.353+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:05.360 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:05.355+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:05.432 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:05.425+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:05.489 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:05.482+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:05.621 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:05.614+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:05.988 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:05.982+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:06.041 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:06.035+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:06.118 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:06.112+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:06.329 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:06.322+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:06.330 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:06.326+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:06.414 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:06.407+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:06.532 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:06.526+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:06.641 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:06.636+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:06.969 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:06.963+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:07.004 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:06.999+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:07.104 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:07.098+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:07.372 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:07.366+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:07.382 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:07.371+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:07.382 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:07.377+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:07.537 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:07.531+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:07.599 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:07.594+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:07.941 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:07.933+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:07.982 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:07.976+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:08.130 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:08.124+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:08.365 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:08.358+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:08.429 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:08.417+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:08.429 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:08.420+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:08.581 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:08.574+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:08.592 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:08.586+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:08.959 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:08.952+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:08.982 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:08.977+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:09.102 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:09.096+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:09.350 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:09.345+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:09.438 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:09.432+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:09.440 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:09.434+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:09.571 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:09.565+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:09.605 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:09.598+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:09.977 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:09.972+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:10.022 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:10.016+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:10.118 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:10.112+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:10.378 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:10.371+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:10.494 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:10.487+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:10.494 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:10.489+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:10.615 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:10.607+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:10.631 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:10.626+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:10.981 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:10.972+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:11.057 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:11.051+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:11.105 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:11.099+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:11.344 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:11.339+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:11.509 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:11.502+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:11.512 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:11.507+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:11.654 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:11.648+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:11.658 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:11.652+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:12.034 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:12.028+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:12.077 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:12.072+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:12.104 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:12.098+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:12.327 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:12.321+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:12.471 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:12.465+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:12.472 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:12.467+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:12.615 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:12.607+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:12.668 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:12.662+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:13.032 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:13.026+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:13.107 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:13.101+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:13.117 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:13.112+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:13.377 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:13.371+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:13.484 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:13.476+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:13.488 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:13.483+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:13.600 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:13.594+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:13.668 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:13.661+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:14.018 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:14.012+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:14.079 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:14.073+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:14.133 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:14.127+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:14.360 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:14.353+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:14.399 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:21:14.395+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e402 v402) 2019-12-09T19:21:14.464 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:14.458+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:14.466 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:14.461+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:14.645 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:14.638+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:14.719 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:14.713+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:15.034 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:15.028+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:15.112 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:15.106+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:15.149 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:15.143+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:15.364 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:15.358+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:15.453 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:15.448+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:15.454 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:15.449+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:15.609 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:15.603+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:15.756 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:15.750+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:16.075 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:16.068+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:16.093 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:16.086+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:16.126 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:16.120+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:16.346 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:16.336+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:16.505 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:16.498+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:16.508 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:16.503+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:16.625 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:16.620+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:16.739 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:16.733+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:17.069 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:17.065+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:17.126 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:17.121+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:17.128 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:17.120+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:17.348 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:17.343+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:17.471 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:17.463+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:17.472 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:17.465+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:17.614 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:17.608+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:17.737 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:17.731+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:18.054 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:18.048+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:18.159 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:18.154+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:18.169 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:18.163+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:18.347 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:18.342+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:18.443 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:18.437+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:18.444 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:18.439+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:18.590 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:18.585+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:18.777 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:18.772+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:19.080 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:19.075+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:19.121 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:19.116+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:19.163 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:19.157+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:19.357 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:19.352+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:19.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:19.412+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:19.417 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:19.413+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:19.421 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:21:19.416+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e402 v402) 2019-12-09T19:21:19.555 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:19.550+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:19.754 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:19.749+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:20.134 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:20.127+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:20.172 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:20.166+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:20.174 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:20.169+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:20.357 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:20.352+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:20.407 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:20.401+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:20.407 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:20.403+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:20.603 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:20.598+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:20.749 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:20.743+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:21.162 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:21.157+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:21.204 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:21.199+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:21.210 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:21.205+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:21.378 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:21.372+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:21.380 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:21.376+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:21.394 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:21.388+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:21.638 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:21.632+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:21.708 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:21.703+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:22.181 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:22.176+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:22.216 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:22.211+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:22.230 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:22.225+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:22.340 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:22.334+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:22.343 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:22.339+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:22.421 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:22.416+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:22.607 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:22.601+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:22.741 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:22.736+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:23.172 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:23.167+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:23.255 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:23.251+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:23.269 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:23.265+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:23.337 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:23.327+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:23.337 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:23.329+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:23.382 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:23.376+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:23.645 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:23.639+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:23.733 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:21:23.728+0000 1694f700 -1 osd.5 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.648011+0000 front 2019-12-09T19:20:02.229588+0000 (oldest deadline 2019-12-09T19:20:43.429636+0000) 2019-12-09T19:21:24.157 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:21:24.152+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.550056+0000 front 2019-12-09T19:20:01.327271+0000 (oldest deadline 2019-12-09T19:20:41.265109+0000) 2019-12-09T19:21:24.254 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:21:24.250+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.653793+0000 front 2019-12-09T19:19:59.647967+0000 (oldest deadline 2019-12-09T19:20:42.518969+0000) 2019-12-09T19:21:24.287 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:21:24.283+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.557868+0000 front 2019-12-09T19:20:00.584459+0000 (oldest deadline 2019-12-09T19:20:40.938423+0000) 2019-12-09T19:21:24.301 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:24.291+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:20:00.167107+0000 front 2019-12-09T19:20:00.615282+0000 (oldest deadline 2019-12-09T19:20:45.683007+0000) 2019-12-09T19:21:24.301 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:24.293+0000 1694f700 -1 osd.0 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:24.397 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:21:24.391+0000 1694f700 -1 osd.2 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:58.209102+0000 front 2019-12-09T19:19:57.650227+0000 (oldest deadline 2019-12-09T19:20:42.919995+0000) 2019-12-09T19:21:24.499 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:21:24.489+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.59:6824/9849,v1:172.21.15.59:6827/9849] for 41sec e402 v402) 2019-12-09T19:21:24.665 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:21:24.660+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.59:6830 osd.3 since back 2019-12-09T19:19:59.649618+0000 front 2019-12-09T19:19:58.309535+0000 (oldest deadline 2019-12-09T19:20:43.470783+0000) 2019-12-09T19:21:25.741 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:25.704+0000 1694f700 -1 osd.0 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:26.706 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:26.670+0000 1694f700 -1 osd.0 404 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:27.796 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:21:27.783+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127725 3.3f 3:fe6d045e:::.dir.478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=149b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T19:21:34.702 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:38:31.321 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.726 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:38:31.344 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.740 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:38:31.359 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.757 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:38:31.376 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.765 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:38:31.364 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.773 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:38:31.372 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.944 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:38:31.563 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:34.958 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:38:31.577 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:21:35.018 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:38:31.617 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:06.734 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:06.725+0000 35276700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:06.977 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:06.963+0000 35a77700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:07.004 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:06.980+0000 3aa81700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:07.162 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:07.078+0000 34274700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:07.482 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:07.279+0000 34274700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:07.794 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:07.570+0000 3727a700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:07.871 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:07.697+0000 3aa81700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:08.208 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:07.808+0000 3aa81700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:10.789 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:10.735+0000 3727a700 -1 osd.3 409 build_incremental_map_msg missing incremental map 409 2019-12-09T19:22:12.224 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:12.221+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:13.229 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:13.222+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:14.252 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:14.248+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:15.261 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:15.251+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:16.267 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:16.263+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:17.290 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:17.286+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:18.248 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:18.243+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:19.208 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:39:15.827 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.223 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:39:15.842 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.225 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:39:15.844 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.238 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:39:15.857 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.241 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:39:15.840 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.246 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:39:15.844 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.263 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:19.259+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:19.355 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:39:15.953 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.387 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:39:16.006 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:19.388 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:39:16.007 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:20.261 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:20.255+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:21.288 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:21.284+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:21.542 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:21.535+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:21.660 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:21.655+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:22.297 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:22.292+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:22.526 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T19:22:22.380+0000 20e25700 -1 RGWWatcher::handle_error cookie 901405120 err (107) Transport endpoint is not connected 2019-12-09T19:22:22.537 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:22.531+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:22.682 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:22.676+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:23.029 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:23.024+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:23.306 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:23.301+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:23.551 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:23.545+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:23.650 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:23.645+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:24.031 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:24.024+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:24.266 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:24.257+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:24.266 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:24.260+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:24.498 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:24.492+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:24.601 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:24.592+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:24.645 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:24.639+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:25.063 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:25.057+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:25.314 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:25.308+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:25.315 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:25.310+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:25.490 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:25.483+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:25.637 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:25.617+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:25.649 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:25.642+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:26.068 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:26.062+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:26.273 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:26.268+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:26.276 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:26.272+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:26.503 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:26.497+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:26.610 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:26.604+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:26.665 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:26.660+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:27.070 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:27.065+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:27.327 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:27.321+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:27.329 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:27.322+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:27.459 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:27.453+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:27.600 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:27.593+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:27.644 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:27.636+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:28.065 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:28.060+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:28.380 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:28.374+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:28.388 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:28.381+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:28.432 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:28.426+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:28.582 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:28.576+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:28.705 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:28.697+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:29.032 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:29.028+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:29.410 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:29.405+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:29.413 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:29.409+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:29.457 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:29.451+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:29.551 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:29.544+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:29.744 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:29.739+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:30.059 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:30.054+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:30.444 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:30.440+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:30.445 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:30.442+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:30.511 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:30.505+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:30.581 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:30.574+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:30.785 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:30.780+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:31.030 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:31.026+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:31.408 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:31.401+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:31.409 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:31.403+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:31.484 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:31.479+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:31.558 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:31.553+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:31.834 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:31.829+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:32.047 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:32.041+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:32.426 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:32.420+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:32.427 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:32.422+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:32.531 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:32.524+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:32.577 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:32.571+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:32.878 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:32.873+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:33.072 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:33.066+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:33.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:33.472+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:33.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:33.474+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:33.507 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:33.501+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:33.612 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:33.606+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:33.891 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:33.886+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:34.046 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:34.041+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:34.519 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:34.512+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:34.519 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:34.514+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:34.562 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:34.555+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:34.627 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:34.621+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:34.910 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:34.905+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:35.039 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:35.034+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:35.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:35.487+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:35.494 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:35.489+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:35.551 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:35.545+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:35.680 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:35.674+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:35.906 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:35.900+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:36.003 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:35.997+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:36.480 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:36.475+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:36.481 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:36.476+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:36.517 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:36.511+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:36.716 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:36.710+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:36.886 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:36.881+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:36.975 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:36.969+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:37.195 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:37.190+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:37.486 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:37.479+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:37.486 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:37.481+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:37.533 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:37.528+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:37.743 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:37.738+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:37.928 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:37.922+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:38.008 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:38.003+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:38.211 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:38.207+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:38.496 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:38.491+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:38.497 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:38.492+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:38.523 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:38.518+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:38.709 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:38.704+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:38.967 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:38.962+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:38.999 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:38.993+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:39.240 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:39.235+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:39.466 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:39.460+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:39.467 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:39.462+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:39.505 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:39.500+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:39.709 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:39.705+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:39.993 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:39.989+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:39.997 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:39.992+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:40.020 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:40.013+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:40.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:40.250+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:40.444 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:40.439+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:40.448 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:40.444+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:40.491 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:40.485+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:40.723 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:40.718+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:40.965 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:40.961+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:40.973 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:40.968+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:40.976 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:40.971+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:41.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:41.252+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:41.436 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:41.429+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:41.437 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:41.432+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:41.538 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:41.533+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:41.684 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:41.679+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:41.952 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:41.948+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:42.018 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:42.012+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:42.023 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:42.018+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:42.228 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:42.221+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:42.232 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:42.226+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:42.472 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:42.467+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:42.477 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:42.470+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:42.554 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:42.549+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:42.679 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:42.674+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:42.929 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:42.923+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:42.985 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:42.981+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:42.987 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:42.980+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:43.201 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:43.196+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:43.202 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:43.197+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:43.478 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:43.472+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:43.479 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:43.474+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:43.535 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:43.529+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:43.668 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:43.663+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:43.970 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:43.964+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:43.989 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:43.984+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:43.999 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:43.993+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:44.200 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:44.195+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:44.201 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:44.196+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:44.493 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:44.488+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:44.496 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:44.491+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:44.544 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:44.538+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:44.659 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:44.654+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:44.961 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:44.956+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:44.985 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:44.979+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:45.000 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:44.995+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:45.153 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:45.148+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:45.154 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:45.149+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:45.505 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:45.500+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:45.506 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:45.502+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:45.536 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:45.530+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:45.657 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:45.651+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:45.951 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:45.946+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:45.965 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:45.956+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:46.007 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:46.003+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:46.191 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:46.185+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:46.192 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:46.187+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:46.515 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:46.509+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:46.516 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:46.511+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:46.551 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:46.544+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:46.668 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:46.662+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:46.917 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:46.912+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:46.918 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:46.913+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:47.057 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:47.052+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:47.164 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:47.156+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:47.164 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:47.157+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:47.507 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:47.500+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:47.508 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:47.502+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:47.517 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:47.511+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:47.715 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:47.710+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:47.887 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:47.882+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:47.923 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:47.911+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:48.107 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:48.101+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:48.146 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:48.140+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:48.146 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:48.142+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:48.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:48.482+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:48.489 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:48.484+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:48.560 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:48.555+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:48.704 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:48.700+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:48.870 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:48.865+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:48.881 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:48.876+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:49.081 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:49.076+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:49.105 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:49.100+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:49.106 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:49.100+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:49.520 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:49.513+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:49.521 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:49.516+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:49.523 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:49.518+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:49.750 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:49.745+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:49.890 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:49.885+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:49.908 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:49.904+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:50.073 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:50.068+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:50.074 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:50.068+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:50.074 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:50.069+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:50.556 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:50.550+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:50.558 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:50.552+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:50.563 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:50.559+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:50.727 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:50.722+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:50.849 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:50.843+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:50.937 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:50.932+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:51.066 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:51.059+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:51.066 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:51.061+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:51.121 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:51.115+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:51.534 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:51.530+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:51.537 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:51.534+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:51.614 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:51.609+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:51.687 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:51.682+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:51.840 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:51.834+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:51.926 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:51.921+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:52.109 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:52.103+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:52.109 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:52.104+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:52.151 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:52.145+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:52.574 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:52.571+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:52.577 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:52.574+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:52.625 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:52.621+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:52.707 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:52.703+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:52.892 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:52.887+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:52.978 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:52.975+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:53.132 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:53.127+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:53.142 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:53.138+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:53.145 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:53.141+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:53.609 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:53.603+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:53.618 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:53.612+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:53.619 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:53.615+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:53.748 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:53.743+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:53.940 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:53.936+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:54.032 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:54.028+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:54.101 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:54.096+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:54.189 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:54.183+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:54.190 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:54.185+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:54.653 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:54.648+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:54.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:54.650+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:54.658 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:54.653+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:54.715 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:54.710+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:54.982 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:54.978+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:55.020 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:55.016+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:55.057 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:22:55.049+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e409 v409) 2019-12-09T19:22:55.120 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:55.115+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:55.202 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:55.197+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:55.203 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:55.198+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:55.614 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:55.608+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:55.615 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:55.611+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:55.619 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:55.614+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:55.718 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:55.713+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:55.960 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:55.956+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:56.071 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:56.066+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:56.136 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:56.127+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:56.188 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:56.183+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:56.189 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:56.184+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:56.612 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:56.606+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:56.644 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:56.640+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:56.648 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:56.645+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:56.757 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:56.752+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:57.007 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:57.003+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:57.036 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:57.031+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:57.127 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:57.118+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:57.202 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:57.198+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:57.204 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:57.200+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:57.618 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:57.613+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:57.619 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:57.615+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:57.628 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:57.623+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:57.781 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:57.776+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:57.997 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:57.987+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:58.012 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:58.007+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:58.084 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:58.079+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:58.181 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:58.175+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:58.182 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:58.177+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:58.609 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:58.604+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:58.625 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:58.620+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:58.627 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:58.622+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:58.765 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:58.760+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:58.977 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:58.973+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:58.986 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:58.982+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:22:59.049 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:22:59.043+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:22:59.190 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:59.185+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:22:59.190 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:22:59.186+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:22:59.621 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:22:59.617+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:22:59.668 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:59.656+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:22:59.669 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:22:59.658+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:22:59.797 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:22:59.792+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:22:59.982 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:22:59.977+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:22:59.983 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:22:59.978+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:00.077 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:23:00.070+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e409 v409) 2019-12-09T19:23:00.096 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:00.090+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:00.153 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:00.146+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:00.154 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:00.150+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:00.637 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:00.631+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:00.712 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:00.706+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:00.713 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:00.708+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:00.810 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:00.806+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:00.966 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:00.962+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:01.016 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:01.012+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:01.135 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:01.130+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:01.203 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:01.197+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:01.204 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:01.199+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:01.641 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:01.635+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:01.687 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:01.682+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:01.688 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:01.683+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:01.852 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:01.846+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:01.920 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:01.916+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:02.038 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:02.032+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:02.099 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:02.095+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:02.214 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:02.209+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:02.215 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:02.211+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:02.635 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:02.630+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:02.704 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:02.697+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:02.705 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:02.699+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:02.884 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:02.879+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:02.900 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:02.894+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:02.998 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:02.993+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:03.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:03.110+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:03.256 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:03.251+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:03.259 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:03.255+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:03.633 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:03.628+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:03.678 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:03.672+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:03.679 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:03.674+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:03.877 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:03.867+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:03.906 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:03.900+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:04.043 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:04.037+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:04.146 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:04.137+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:04.231 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:04.226+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:04.234 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:04.230+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:04.653 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:04.648+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:04.659 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:04.652+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:04.678 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:04.674+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:04.832 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:04.827+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:04.917 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:04.912+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:05.089 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:23:05.084+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e409 v409) 2019-12-09T19:23:05.092 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:05.087+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:05.114 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:05.109+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:05.271 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:05.266+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:05.272 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:05.267+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:05.657 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:05.651+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:05.679 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:05.673+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:05.680 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:05.675+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:05.824 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:05.819+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:05.917 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:05.912+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:06.065 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:06.060+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:06.149 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:06.143+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:06.319 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:06.314+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:06.322 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:06.318+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:06.681 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:06.670+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:06.693 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:06.687+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:06.695 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:06.689+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:06.818 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:06.813+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:06.933 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:06.927+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:07.064 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:07.058+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:07.158 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:07.148+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:07.337 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:07.329+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:07.337 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:07.331+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:07.703 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:07.697+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:07.708 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:07.702+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:07.726 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:07.720+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:07.773 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:07.767+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:07.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:07.968+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:08.029 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:08.024+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:08.125 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:08.117+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:08.320 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:08.313+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:08.320 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:08.314+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:08.721 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:08.717+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:08.723 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:08.719+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:08.747 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:08.742+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:08.826 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:08.819+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:08.960 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:08.954+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:09.003 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:08.997+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:09.110 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:09.104+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:09.371 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:09.364+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:09.372 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:09.366+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:09.705 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:09.700+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:09.706 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:09.701+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:09.732 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:09.727+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:09.812 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:09.806+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:10.006 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:09.999+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:10.006 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:10.001+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:10.109 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:23:10.100+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e409 v409) 2019-12-09T19:23:10.120 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:10.115+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:10.347 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:10.342+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:10.349 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:10.344+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:10.710 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:10.704+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:10.711 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:10.706+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:10.742 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:10.737+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:10.796 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:10.791+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:11.014 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:11.010+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:11.028 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:11.022+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:11.116 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:11.110+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:11.374 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:11.365+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:11.381 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:11.367+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:11.674 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:11.667+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:11.675 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:11.669+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:11.783 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:11.778+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:11.790 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:11.782+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:12.040 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:12.035+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:12.063 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:12.058+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:12.076 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:12.069+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:12.380 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:12.372+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:12.381 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:12.374+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:12.650 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:12.645+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:12.654 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:12.650+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:12.766 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:12.761+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:12.830 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:12.824+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:13.041 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:13.034+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:13.047 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:13.043+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:13.100 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:13.094+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:13.425 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:13.417+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:13.425 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:13.419+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:13.683 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:13.678+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:13.687 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:13.682+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:13.784 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:13.779+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:13.856 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:13.850+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:14.056 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:14.050+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:14.056 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:14.051+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:14.079 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:14.074+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:14.462 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:14.456+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:14.463 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:14.457+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:14.666 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:14.660+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:14.670 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:14.665+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:14.744 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:14.738+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:14.859 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:14.853+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:15.013 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:15.007+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:15.044 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:15.038+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:15.085 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:15.080+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:15.129 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:23:15.116+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e409 v409) 2019-12-09T19:23:15.470 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:15.465+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:15.472 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:15.468+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:15.672 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:15.666+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:15.673 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:15.668+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:15.698 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:15.693+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:15.898 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:15.891+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:16.005 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:16.000+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:16.028 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:16.020+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:16.113 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:16.108+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:16.460 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:16.454+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:16.464 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:16.458+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:16.670 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:16.664+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:16.710 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:16.704+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:16.711 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:16.706+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:16.886 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:16.880+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:17.012 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:17.006+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:17.052 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:17.046+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:17.107 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:17.101+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:17.491 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:17.484+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:17.492 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:17.486+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:17.649 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:17.644+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:17.756 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:17.752+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:17.760 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:17.756+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:17.881 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:17.874+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:18.039 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:18.034+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:18.063 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:18.056+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:18.065 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:18.060+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:18.493 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:18.486+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:18.494 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:18.488+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:18.667 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:18.662+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:18.799 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:18.793+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:18.801 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:18.795+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:18.877 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:18.870+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:19.014 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:19.006+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:19.030 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:19.024+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:19.079 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:19.074+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:19.517 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:19.512+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:19.520 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:19.515+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:19.704 INFO:tasks.ceph.osd.5.smithi161.stderr:2019-12-09T19:23:19.698+0000 1694f700 -1 osd.5 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:43.349624+0000 front 2019-12-09T19:21:42.466290+0000 (oldest deadline 2019-12-09T19:22:22.398310+0000) 2019-12-09T19:23:19.837 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:19.830+0000 1694f700 -1 osd.4 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:39.522854+0000 front 2019-12-09T19:21:39.563395+0000 (oldest deadline 2019-12-09T19:22:24.178647+0000) 2019-12-09T19:23:19.838 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:19.832+0000 1694f700 -1 osd.4 409 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:19.905 INFO:tasks.ceph.osd.2.smithi059.stderr:2019-12-09T19:23:19.898+0000 1694f700 -1 osd.2 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.882768+0000 front 2019-12-09T19:21:42.998795+0000 (oldest deadline 2019-12-09T19:22:23.671033+0000) 2019-12-09T19:23:19.998 INFO:tasks.ceph.osd.0.smithi059.stderr:2019-12-09T19:23:19.993+0000 1694f700 -1 osd.0 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.087437+0000 front 2019-12-09T19:21:42.089321+0000 (oldest deadline 2019-12-09T19:22:39.067702+0000) 2019-12-09T19:23:20.017 INFO:tasks.ceph.osd.1.smithi059.stderr:2019-12-09T19:23:20.012+0000 1694f700 -1 osd.1 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.472986+0000 front 2019-12-09T19:21:42.471654+0000 (oldest deadline 2019-12-09T19:22:21.349608+0000) 2019-12-09T19:23:20.056 INFO:tasks.ceph.osd.7.smithi161.stderr:2019-12-09T19:23:20.052+0000 1694f700 -1 osd.7 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 since back 2019-12-09T19:21:42.471404+0000 front 2019-12-09T19:21:41.595764+0000 (oldest deadline 2019-12-09T19:22:21.297330+0000) 2019-12-09T19:23:20.201 INFO:tasks.ceph.mon.a.smithi059.stderr:2019-12-09T19:23:20.193+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.161:6811/9942,v1:172.21.15.161:6813/9942] for 42sec e409 v409) 2019-12-09T19:23:20.481 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:20.475+0000 1694f700 -1 osd.3 409 heartbeat_check: no reply from 172.21.15.161:6823 osd.6 ever on either front or back, first ping sent 2019-12-09T19:22:01.328178+0000 (oldest deadline 2019-12-09T19:22:41.328178+0000) 2019-12-09T19:23:20.481 INFO:tasks.ceph.osd.3.smithi059.stderr:2019-12-09T19:23:20.476+0000 1694f700 -1 osd.3 409 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4367.0:130599 7.7 7.12a65e07 (undecoded) ondisk+retry+read+known_if_redirected e407) 2019-12-09T19:23:21.176 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:21.017+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:22.166 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:23:22.161+0000 1694f700 -1 osd.4 411 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4361.0:127893 2.6s0 2:63841728:::478c2bee-74bb-4cd4-b56a-b2f2fa8233cf.4367.633_%2ffile A:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21 in=21b,setxattr user.rgw.manifest (336) in=353b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (22) in=43b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-12-09T19:23:50.001 INFO:tasks.ceph.osd.6.smithi161.stderr:*** Caught signal (Segmentation fault) ** 2019-12-09T19:23:50.001 INFO:tasks.ceph.osd.6.smithi161.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-09T19:23:50.066 INFO:tasks.ceph.osd.6.smithi161.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T19:23:50.066 INFO:tasks.ceph.osd.6.smithi161.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T19:23:50.066 INFO:tasks.ceph.osd.6.smithi161.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T19:23:50.067 INFO:tasks.ceph.osd.6.smithi161.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T19:23:50.067 INFO:tasks.ceph.osd.6.smithi161.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T19:23:50.067 INFO:tasks.ceph.osd.6.smithi161.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T19:23:50.067 INFO:tasks.ceph.osd.6.smithi161.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T19:23:50.068 INFO:tasks.ceph.osd.6.smithi161.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T19:23:50.068 INFO:tasks.ceph.osd.6.smithi161.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T19:23:50.068 INFO:tasks.ceph.osd.6.smithi161.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T19:23:50.068 INFO:tasks.ceph.osd.6.smithi161.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T19:23:50.069 INFO:tasks.ceph.osd.6.smithi161.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T19:23:50.069 INFO:tasks.ceph.osd.6.smithi161.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T19:23:50.069 INFO:tasks.ceph.osd.6.smithi161.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T19:23:50.147 INFO:tasks.ceph.osd.6.smithi161.stderr:2019-12-09T19:23:50.139+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T19:23:50.147 INFO:tasks.ceph.osd.6.smithi161.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-09T19:23:50.148 INFO:tasks.ceph.osd.6.smithi161.stderr: 2019-12-09T19:23:50.148 INFO:tasks.ceph.osd.6.smithi161.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T19:23:50.148 INFO:tasks.ceph.osd.6.smithi161.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T19:23:50.148 INFO:tasks.ceph.osd.6.smithi161.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T19:23:50.149 INFO:tasks.ceph.osd.6.smithi161.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T19:23:50.149 INFO:tasks.ceph.osd.6.smithi161.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T19:23:50.149 INFO:tasks.ceph.osd.6.smithi161.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T19:23:50.149 INFO:tasks.ceph.osd.6.smithi161.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T19:23:50.150 INFO:tasks.ceph.osd.6.smithi161.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T19:23:50.150 INFO:tasks.ceph.osd.6.smithi161.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T19:23:50.150 INFO:tasks.ceph.osd.6.smithi161.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T19:23:50.152 INFO:tasks.ceph.osd.6.smithi161.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T19:23:50.152 INFO:tasks.ceph.osd.6.smithi161.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T19:23:50.152 INFO:tasks.ceph.osd.6.smithi161.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T19:23:50.153 INFO:tasks.ceph.osd.6.smithi161.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T19:23:50.153 INFO:tasks.ceph.osd.6.smithi161.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T19:23:50.153 INFO:tasks.ceph.osd.6.smithi161.stderr: 2019-12-09T19:23:50.738 INFO:tasks.ceph.osd.6.smithi161.stderr: -2> 2019-12-09T19:23:50.139+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T19:23:50.738 INFO:tasks.ceph.osd.6.smithi161.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-09T19:23:50.738 INFO:tasks.ceph.osd.6.smithi161.stderr: 2019-12-09T19:23:50.738 INFO:tasks.ceph.osd.6.smithi161.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T19:23:50.738 INFO:tasks.ceph.osd.6.smithi161.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T19:23:50.739 INFO:tasks.ceph.osd.6.smithi161.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T19:23:50.739 INFO:tasks.ceph.osd.6.smithi161.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T19:23:50.739 INFO:tasks.ceph.osd.6.smithi161.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T19:23:50.739 INFO:tasks.ceph.osd.6.smithi161.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T19:23:50.740 INFO:tasks.ceph.osd.6.smithi161.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T19:23:50.740 INFO:tasks.ceph.osd.6.smithi161.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T19:23:50.740 INFO:tasks.ceph.osd.6.smithi161.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T19:23:50.740 INFO:tasks.ceph.osd.6.smithi161.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T19:23:50.740 INFO:tasks.ceph.osd.6.smithi161.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T19:23:50.741 INFO:tasks.ceph.osd.6.smithi161.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T19:23:50.741 INFO:tasks.ceph.osd.6.smithi161.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T19:23:50.741 INFO:tasks.ceph.osd.6.smithi161.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T19:23:50.741 INFO:tasks.ceph.osd.6.smithi161.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T19:23:50.742 INFO:tasks.ceph.osd.6.smithi161.stderr: 2019-12-09T19:23:51.091 INFO:tasks.ceph.osd.6.smithi161.stderr: -1003> 2019-12-09T19:23:50.139+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T19:23:51.091 INFO:tasks.ceph.osd.6.smithi161.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-09T19:23:51.091 INFO:tasks.ceph.osd.6.smithi161.stderr: 2019-12-09T19:23:51.091 INFO:tasks.ceph.osd.6.smithi161.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T19:23:51.091 INFO:tasks.ceph.osd.6.smithi161.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T19:23:51.092 INFO:tasks.ceph.osd.6.smithi161.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T19:23:51.092 INFO:tasks.ceph.osd.6.smithi161.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T19:23:51.092 INFO:tasks.ceph.osd.6.smithi161.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T19:23:51.092 INFO:tasks.ceph.osd.6.smithi161.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T19:23:51.092 INFO:tasks.ceph.osd.6.smithi161.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T19:23:51.092 INFO:tasks.ceph.osd.6.smithi161.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T19:23:51.093 INFO:tasks.ceph.osd.6.smithi161.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T19:23:51.094 INFO:tasks.ceph.osd.6.smithi161.stderr: 2019-12-09T19:23:54.132 INFO:tasks.ceph.osd.6.smithi161.stderr:daemon-helper: command crashed with signal 11 2019-12-09T19:23:58.364 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2019-12-09T19:24:04.677 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2019-12-09T19:24:10.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~13s 2019-12-09T19:24:17.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~19s 2019-12-09T19:24:23.614 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2019-12-09T19:24:29.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~32s 2019-12-09T19:24:36.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~38s 2019-12-09T19:24:42.550 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~44s 2019-12-09T19:24:48.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~50s 2019-12-09T19:24:55.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~57s 2019-12-09T19:25:01.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~63s 2019-12-09T19:25:07.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~69s 2019-12-09T19:25:14.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~76s 2019-12-09T19:25:20.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~82s 2019-12-09T19:25:26.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~88s 2019-12-09T19:25:33.052 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2019-12-09T19:25:39.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~101s 2019-12-09T19:25:45.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~107s 2019-12-09T19:25:52.007 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~114s 2019-12-09T19:25:58.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2019-12-09T19:26:04.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~126s 2019-12-09T19:26:10.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~133s 2019-12-09T19:26:17.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~139s 2019-12-09T19:26:23.579 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~145s 2019-12-09T19:26:29.890 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~152s 2019-12-09T19:26:36.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~158s 2019-12-09T19:26:39.554 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:43:36.173 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.558 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:43:36.178 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.577 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:43:36.196 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.585 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:43:36.204 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.587 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:43:36.186 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.629 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:43:36.227 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.843 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:43:36.462 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.850 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:43:36.469 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:39.950 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:43:36.549 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~164s 2019-12-09T19:26:48.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2019-12-09T19:26:55.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~177s 2019-12-09T19:27:01.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~183s 2019-12-09T19:27:07.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~189s 2019-12-09T19:27:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~196s 2019-12-09T19:27:20.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~202s 2019-12-09T19:27:26.695 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~208s 2019-12-09T19:27:33.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2019-12-09T19:27:39.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~221s 2019-12-09T19:27:45.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~227s 2019-12-09T19:27:51.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~234s 2019-12-09T19:27:58.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2019-12-09T19:28:04.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~246s 2019-12-09T19:28:10.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~253s 2019-12-09T19:28:17.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~259s 2019-12-09T19:28:23.520 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2019-12-09T19:28:29.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~271s 2019-12-09T19:28:36.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~278s 2019-12-09T19:28:42.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~284s 2019-12-09T19:28:48.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~290s 2019-12-09T19:28:55.093 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~297s 2019-12-09T19:29:01.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~303s 2019-12-09T19:29:01.406 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-12-09T19:29:01.406 ERROR:tasks.daemonwatchdog.daemon_watchdog:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 38, in _run self.watch() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 114, in watch self.bark() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 54, in bark for mount in self.ctx.mounts.values(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/config.py", line 253, in __getattr__ raise AttributeError(name) AttributeError: mounts 2019-12-09T19:29:56.863 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:46:53.483 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:56.908 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:46:53.527 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:56.929 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:46:53.548 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:56.934 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:46:53.553 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:56.997 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:46:53.596 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:57.000 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:46:53.599 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:57.051 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:46:53.670 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:57.053 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:46:53.672 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:29:57.233 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:46:53.833 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:46.986 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:48:43.605 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:46.992 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:48:43.611 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.057 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:48:43.653 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.078 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:48:43.697 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.081 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:48:43.700 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.086 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:48:43.684 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.095 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:48:43.714 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.195 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:48:43.812 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:47.317 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:48:43.914 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:33.597 INFO:tasks.rgw.client.0.smithi059.stdout:2019-12-09T19:34:33.517+0000 20e25700 -1 RGWWatcher::handle_error cookie 5397776432 err (110) Connection timed out 2019-12-09T19:34:36.775 INFO:tasks.ceph.osd.4.smithi161.stderr:2019-12-09T19:34:36.768+0000 3aa81700 -1 osd.4 426 build_incremental_map_msg missing incremental map 393 2019-12-09T19:36:11.941 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:53:08.556 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:11.947 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:53:08.566 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:11.969 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:53:08.586 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:11.971 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:53:08.591 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:11.988 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:53:08.573 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:11.989 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:53:08.583 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:12.230 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:53:08.850 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:12.257 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:53:08.876 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:12.370 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:53:08.969 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.138 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:53:40.757 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.153 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:53:40.769 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.205 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:53:40.824 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.221 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:53:40.827 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.221 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:53:40.837 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.261 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:53:40.857 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.265 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:53:40.862 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.265 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:53:40.881 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:44.341 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:53:40.934 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.539 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:55:16.158 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.543 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:55:16.162 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.628 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:55:16.247 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.631 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:55:16.250 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.637 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:55:16.237 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.640 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:55:16.239 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.681 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:55:16.300 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.700 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:55:16.319 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:19.728 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:55:16.328 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.443 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:58:18.062 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.455 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:58:18.075 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.472 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:58:18.091 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.472 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:58:18.068 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.475 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:58:18.095 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.481 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:58:18.079 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.597 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:09:58:18.216 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.641 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:09:58:18.231 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:21.641 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:09:58:18.259 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:33.746 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:03:30.365 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:33.766 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:03:30.385 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:33.770 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:03:30.389 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:33.777 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:03:30.396 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:33.788 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:03:30.381 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:33.820 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:03:30.418 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:34.018 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:03:30.635 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:34.105 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:03:30.724 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:34.142 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:03:30.742 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.640 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:03:38.260 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.645 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:03:38.265 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.690 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:03:38.310 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.692 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:03:38.311 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.706 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:03:38.325 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.721 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:03:38.338 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.737 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:03:38.336 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.741 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:03:38.341 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:41.779 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:03:38.377 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.007 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:08:42.626 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.020 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:08:42.638 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.031 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:08:42.651 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.038 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:08:42.658 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.041 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:08:42.641 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.050 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:08:42.650 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.238 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:08:42.857 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.250 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:08:42.849 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:46.256 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:08:42.876 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.532 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:12:00.151 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.562 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:12:00.181 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.563 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:12:00.183 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.594 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:12:00.213 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.620 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:12:00.220 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.625 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:12:00.225 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.668 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:12:00.288 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.696 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:12:00.316 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:03.769 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:12:00.369 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.344 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:13:54.963 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.356 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:13:54.973 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.379 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:13:54.998 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.383 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:13:55.003 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.400 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:13:55.000 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.406 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:13:55.006 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.467 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:13:55.087 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.486 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:13:55.105 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:58.681 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:13:55.282 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.338 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:19:07.957 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.347 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:19:07.961 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.357 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:19:07.976 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.362 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:19:07.981 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.380 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:19:07.980 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.403 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:19:08.003 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.573 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:19:08.193 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.579 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:19:08.199 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:11.673 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:19:08.273 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.474 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:20:22.092 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.511 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:20:22.130 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.548 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:20:22.168 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.551 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:20:22.171 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.552 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:20:22.151 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.554 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:20:22.154 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.591 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:20:22.210 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.622 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:20:22.223 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:03:25.644 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:20:22.263 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:22.852 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:24:19.471 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:22.903 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:24:19.485 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:22.903 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:24:19.496 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:22.903 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:24:19.477 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:22.904 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:24:19.490 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:22.904 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:24:19.495 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:23.035 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:24:19.654 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:23.051 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:24:19.670 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:23.135 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:24:19.735 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.381 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:28:43.994 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.436 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:28:44.055 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.440 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:28:44.059 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.441 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:28:44.060 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.529 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:28:44.129 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.536 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:28:44.137 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.588 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:28:44.208 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.604 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:28:44.222 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:47.691 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:28:44.291 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.629 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:29:29.248 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.637 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:29:29.255 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.652 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:29:29.270 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.662 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:29:29.276 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.663 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:29:29.263 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.680 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:29:29.280 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.697 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:29:29.316 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.728 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:29:29.347 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:32.830 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:29:29.429 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.679 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:34:40.297 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.687 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:34:40.306 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.692 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:34:40.312 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.695 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:34:40.315 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.705 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:34:40.305 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.709 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:34:40.310 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.936 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:34:40.555 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:43.938 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:34:40.558 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:44.015 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:34:40.613 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:53.952 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:34:50.571 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:53.997 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:34:50.616 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.026 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:34:50.645 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.031 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:34:50.650 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.045 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:34:50.665 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.056 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:34:50.655 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.063 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:34:50.663 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.068 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:34:50.688 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:54.096 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:34:50.697 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.610 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:37:06.229 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.621 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:37:06.240 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.690 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:37:06.309 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.696 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:37:06.315 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.712 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:37:06.311 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.714 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:37:06.314 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.735 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:37:06.354 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.814 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:37:06.433 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:20:09.826 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:37:06.426 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.502 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:39:50.122 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.511 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:39:50.130 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.521 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:39:50.140 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.527 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:39:50.146 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.539 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:39:50.139 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.563 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:39:50.163 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.681 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:39:50.301 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.714 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:39:50.333 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:53.733 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:39:50.333 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.488 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:45:03.107 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.496 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:45:03.116 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.514 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:45:03.134 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.523 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:45:03.143 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.525 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:45:03.125 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.531 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:45:03.132 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.736 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:45:03.356 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.752 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:45:03.371 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:06.783 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:45:03.383 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.478 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:45:28.098 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.507 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:45:28.126 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.529 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:45:28.141 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.532 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:45:28.152 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.533 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:45:28.133 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.545 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:45:28.165 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.573 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:45:28.193 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.577 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:45:28.177 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:28:31.625 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:45:28.226 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.722 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:50:15.342 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.734 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:50:15.349 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.749 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:50:15.369 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.751 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:50:15.352 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.761 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:50:15.381 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.762 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:50:15.362 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.915 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:50:15.535 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:18.949 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:50:15.569 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:19.134 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:50:15.735 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.434 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:53:50.054 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.483 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:53:50.103 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.520 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:53:50.120 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.523 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:53:50.124 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.525 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:53:50.146 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.528 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:53:50.148 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.612 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:53:50.232 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.686 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:53:50.306 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:36:53.710 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:53:50.311 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.285 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:55:26.903 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.290 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:55:26.908 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.302 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:55:26.922 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.305 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:55:26.925 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.307 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:55:26.908 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.313 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:55:26.914 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.401 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:10:55:27.021 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.401 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:10:55:27.021 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:30.443 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:10:55:27.043 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:39.988 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:00:36.609 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.017 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:00:36.627 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.017 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:00:36.631 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.017 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:00:36.638 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.027 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:00:36.628 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.047 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:00:36.648 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.239 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:00:36.860 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.263 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:00:36.884 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:40.362 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:00:36.963 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.302 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:02:11.920 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.308 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:02:11.928 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.343 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:02:11.964 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.353 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:02:11.973 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.382 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:02:12.003 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.409 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:02:12.009 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.415 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:02:12.016 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.418 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:02:12.038 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:15.502 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:02:12.103 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.289 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:05:47.909 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.299 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:05:47.920 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.317 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:05:47.938 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.323 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:05:47.924 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.327 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:05:47.949 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.342 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:05:47.942 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.490 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:05:48.111 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.527 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:05:48.128 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:51.528 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:05:48.149 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.389 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:10:34.010 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.399 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:10:34.020 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.450 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:10:34.071 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.458 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:10:34.079 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.497 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:10:34.090 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.498 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:10:34.093 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.587 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:10:34.208 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.637 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:10:34.251 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:37.729 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:10:34.329 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.597 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:10:59.218 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.603 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:10:59.224 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.630 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:10:59.231 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.644 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:10:59.265 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.646 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:10:59.267 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.649 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:10:59.270 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.676 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:10:59.276 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.688 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:10:59.309 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:54:02.779 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:10:59.380 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:03.850 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:16:00.472 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:03.869 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:16:00.490 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:03.900 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:16:00.521 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:03.908 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:16:00.530 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:03.936 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:16:00.536 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:03.940 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:16:00.541 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:04.062 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:16:00.683 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:04.109 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:16:00.731 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:04.157 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:16:00.756 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.038 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:16:10.659 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.047 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:16:10.669 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.062 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:16:10.663 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.067 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:16:10.668 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.069 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:16:10.690 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.074 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:16:10.695 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.106 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:16:10.727 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.108 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:16:10.730 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:14.119 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:16:10.720 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.504 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:18:56.126 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.556 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:18:56.178 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.558 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:18:56.180 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.559 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:18:56.180 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.601 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:18:56.202 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.606 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:18:56.207 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.648 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:18:56.270 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.675 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:18:56.296 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:01:59.723 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:18:56.324 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.172 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:21:22.793 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.180 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:21:22.801 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.205 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:21:22.826 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.205 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:21:22.806 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.210 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:21:22.832 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.239 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:21:22.840 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.383 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:21:23.004 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.387 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:21:23.007 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:26.424 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:21:23.025 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:39.918 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:26:36.539 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:39.927 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:26:36.547 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:39.936 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:26:36.558 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:39.941 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:26:36.563 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:39.942 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:26:36.543 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:39.973 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:26:36.574 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:40.146 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:26:36.768 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:40.150 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:26:36.772 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:09:40.235 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:26:36.836 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.394 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:27:18.016 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.405 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:27:18.026 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.441 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:27:18.063 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.478 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:27:18.099 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.483 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:27:18.105 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.495 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:27:18.096 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.498 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:27:18.099 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.517 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:27:18.139 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:21.642 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:27:18.242 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.178 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:31:47.800 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.193 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:31:47.808 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.195 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:31:47.817 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.198 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:31:47.821 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.209 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:31:47.811 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.231 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:31:47.833 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.375 INFO:tasks.ceph.mon.c.smithi059.stderr:==00:11:31:47.997 8892== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.416 INFO:tasks.ceph.mon.a.smithi059.stderr:==00:11:31:48.038 8894== Warning: unimplemented fcntl command: 1036 2019-12-09T21:14:51.490 INFO:tasks.ceph.mon.b.smithi161.stderr:==00:11:31:48.091 8631== Warning: unimplemented fcntl command: 1036 2019-12-09T21:17:24.346 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-12-09T21:17:24.346 DEBUG:teuthology.task.console_log:Killing console logger for smithi161 2019-12-09T21:17:24.347 DEBUG:teuthology.task.console_log:Killing console logger for smithi059 2019-12-09T21:17:24.347 DEBUG:teuthology.task.console_log:Killing console logger for smithi161 2019-12-09T21:17:24.347 DEBUG:teuthology.task.console_log:Killing console logger for smithi059 2019-12-09T21:17:24.348 DEBUG:teuthology.exit:Finished running handlers